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Abstract 


Now is an opportune time to consider new ways to ask registrants for the GRE ® General Test 
about their race and ethnicity. Growth in the percentage of Americans who identify as multiracial 
suggests the possibility that a sizeable percentage of registrants would self-report more than one 
race or ethnicity if given the opportunity to do so; furthermore, racial and ethnic questions 
currently asked might be inappropriate or insufficient for the increasing number of registrants 
who are not U.S. citizens or pennanent U.S. residents and continuing to consider Hispanic origin 
a racial classification might be too restrictive. Because fairness at the group level can reflect only 
the groups that can be identified, how we ask people to identify their race and ethnicity is 
fundamental. Using stratified sampling and online questionnaires, we explored U.S.-style and 
international-style questions about race and ethnicity as potential replacements of or supplements 
to the questions on ETS’s current GRE Background Infonnation Questionnaire (BIQ). We 
investigated the effects of U.S. citizenship, birth country, U.S. pennanent residency status, 
mono- and multiraciality, and Hispanic origin on the preferences that survey respondents had for 
certain question fonnats and content. Furthermore, we analyzed the qualitative survey responses 
to determine respondents’ major concerns about various question formats. In response to the 
survey results, we recommend that ETS’s current BIQ be amended to include (a) more pluralistic 
U.S.-style questions about race and ethnicity that conform to U.S. federal standards and 
(b) certain international-style questions. 

Key words: race, ethnicity, diversity, demographics, monoracial, multiracial, Background 
Infonnation Questionnaire, BIQ, 1997 Standards, differential item functioning, DIF, differential 
prediction, predictive bias, fairness 
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Executive Summary 

The population of those who seek graduate or professional education in programs that use 
the GRE ® General Test (now the GRE revised General Test) is changing substantially. Among 
other things, there are more multiracial and international registrants than in the past. It appears 
that this trend will continue into the future. In addition, the legal landscape has changed. U.S. 
federal regulations now require certain institutions to collect and report racial and ethnic 
infonnation using prescribed formats and content (e.g., Final Guidance on Maintaining, 
Collecting, and Reporting Racial and Ethnic Data to the U.S. Department of Education, 2007). 
Even if the regulations and other laws do not directly govern an organization, there may be 
prudent reasons to follow them anyway. 

In recognition of these changes, ETS has undertaken a study to examine if and how it 
should change the current questions about race and ethnicity on its GRE Background 
Infonnation Questionnaire (BIQ). This report explains some of the implications of how questions 
about race and ethnicity are asked, as well as describes an investigation of U.S.-style and 
international-style questions. Our major findings are as follows: 

• U.S. acculturation (being a U.S. citizen, being born in the United States, and/or being 
a pennanent U.S. resident) was related to a preference for a more pluralistic racial and 
ethnic question fonnat that pennits multiracial self-reporting and separate questions 
about Hispanic origin and race. 

• Registrants who are not acculturated to U.S. conceptualizations of race and ethnicity 
generally preferred a simpler design like the one in ETS’s current GRE BIQ. 

• Regardless of one’s acculturation, being monoracial was related to being inclined 
toward the simpler format, whereas being multiracial was related to being inclined 
toward a more pluralistic format (i.e., one that recognizes more complex diversity). 

• Many registrants who prefer a more pluralistic question do so because they want to be 
able to identify their race and ethnicity more specifically, such as in the form of 
national origin or tribal affiliation. 

• However, Hispanic respondents often did not want racial and ethnic choices in 
addition to those for Hispanic origin. In fact, they often expressed confusion about 
how to report their race. 
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• There was an overall desire for questions that permit reporting of more than one race 
as well as national origin/ancestry (especially when the respondent’s national 
origin/ancestry was not provided as an option). 

• In general, the international respondents did not express objections to the 
international-style questions on the survey; however, some of their comments (as well 
as their expressed question preferences) indicated that ethnicity, race, and citizenship 
have complex and varying relationships that U.S.-style questions about race and 
ethnicity may not capture well or at all outside of the United States. 

In response to the survey results, we recommend that ETS’s current BIQ be amended to 
include (a) certain U.S.-style questions about race and ethnicity that conform to the 1997 
Standards and are more pluralistic as well as (b) certain international-style questions. 

Overview 

ETS’s collection of racial and ethnic information from the users of its assessments 
permits ETS to conduct differential item functioning (DIF) and differential prediction (DP) 
analyses. DIF occurs when people from different groups (racial, ethnic, etc.) but of the same 
level of ability (or other trait) that is being measured by a test item are not equally likely to 
answer that item correctly (or the same; see Dorans & Holland, 1993). DP, occasionally referred 
to as predictive bias, occurs when an assessment predicts an outcome criterion (e.g., future GPA) 
differently for different groups (see Kuncel & Klieger, 2012). Sometimes DIF and DP analyses 
will be referred to in this report as types of fairness analyses, because they detect situations when 
a difference in group membership that is unrelated to a predictor (a test) or outcome criterion is 
nevertheless related to measurement on that predictor or outcome criterion. In addition, ETS’s 
collection of racial and ethnic information could allow ETS to detennine the extent to which 
certain groups may be under- or overrepresented among those who consider and those who apply 
to graduate and professional schools that use the GRE revised General Test. Moreover, the 
collection and reporting of racial and ethnic information might be required legally or 
contractually as part of ETS’s work for governmental and other organizations. 

Thus, there are important reasons for ETS to collect racial and ethnic information from 
the users of its assessments. How ETS elicits infonnation about race and ethnicity is an integral 
but distinct issue that may affect, among other things: 
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1. whether ETS is in compliance with applicable U.S. federal regulations (or other 
federal, state, local, international, and foreign laws); 

2. what ETS can know about DIF and DP for the GRE General Test; 

3. the accuracy of responses to race and ethnicity questions (i.e., the extent to which 
registrants understand and produce the kind of responses that the questions are 
intended to elicit); and 

4. registrants’ comfort level with the race and ethnicity questions and consequently their 
opinions about their experience with the assessment (and its developer, ETS). 

The way in which ETS elicits racial and ethnic information might need to conform to 
various laws. Notably, the U.S. federal Office of Management and Budget (OMB) issued a 
government-wide directive in 1997, sometimes referred to as the 1997 Standards or the 
Revisions to Directive 15 (Revisions to the Standards for the Classification of Federal Data on 
Race and Ethnicity, 1997). The 1997 Standards established the parameters for the questions 
about race and ethnicity in the 2000 U.S. Census questionnaire and the surveys, fonns, and 
records that have been used by the U.S. federal government’s departments and agencies as of 
January 2003. This includes the U.S. Department of Education (DOE), which issued its own final 
guidance for compliance with the 1997 Standards in 2007 (Final Guidance on Maintaining, 
Collecting, and Reporting Racial and Ethnic Data to the U.S. Department of Education, 2007). 
The DOE instructed educational institutions to follow the 1997 Standards by the fall of 2010. If, 
and the extent to which, these or other regulations and laws apply to ETS may depend upon 
whether ETS or one of its programs has or intends to seek government contracts, funding, 
assistance, or endorsement. ETS research sponsors or participants, as well as ETS stakeholders, 
might be legally bound by the 1997 Standards or other data collection and reporting laws. 
Therefore, it might be practical for ETS to abide by those laws even if they do not directly 
govern data collection or reporting by ETS. 

How ETS collects racial and ethnic information detennines what ETS can know about 
DIF and DP for the GRE General Test. More information about race and ethnicity pennits more 
detailed DIF and DP analyses, although one could reasonably argue that more infonnation is not 
necessarily better. The division of registrants into additional demographic groups reduces sample 
sizes and thus increases the risk of sampling error. 
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Sampling error increases or decreases true effect sizes by random magnitudes. The more 
often one conducts comparisons, the more likely that one of these comparisons will erroneously 
result in an observed effect size that is large enough to be considered practically significant when 
the true effect size would not be. In addition, as long as sample sizes and risk tolerance are large 
enough, there is no clear scientific basis for knowing when to stop the process of dividing people 
into additional categories for purposes of DIF or DP analyses. With more groups, the choices 
about whether certain groups should be combined or treated separately become more numerous. 
On the other hand, it is impossible to detect even large and enduring effect sizes without 
sufficient information. An assessment may function or predict differently for two groups, but if 
one lacks the data to create these groups, then the differential functioning or prediction will 
continue undetected. Also, one cannot detennine the risk of sampling error or the number of DIF 
and DP analyses that could feasibly be run without at least some exploration into uncharted 
territory. 


The 1997 Standards and 2010 U.S. Census Questionnaire 

Because we did not possess actual test data (GRE scores), we could not directly address 
the impact of question fonnat and content on DIF or DP analyses. However, we could explore 
how question format and content would affect the accuracy of racial and ethnic responses in 
terms of whether registrants understand what is being asked as well as their comfort levels with 
these questions. In particular, we explored the racial and ethnic categorizations that the 1997 
Standards and 2010 U.S. Census questionnaire facilitate. As discussed, ETS’s compliance with 
the 1997 Standards and the U.S. Census content and fonnat might at least be prudent (if not 
required). While the 1997 Standards and the U.S. Census content and format are in the end based 
on debatable value judgments, they nevertheless have a thoughtful and sizeable research basis 
(see, e.g., Alberti, 2006; Martin, 2006, 2007; Martin, Sheppard, Bentley, & Bennett, 2007; 
Revisions to the Standards for the Classification of Federal Data on Race and Ethnicity, 1997; 
U.S. Census Bureau, 2006). Although it might not have been the specific intent of their creators, 
the 1997 Standards and the U.S. Census form also address major changes in the population that 
seeks higher education in the United States, including a substantial increase in multiracial 
applicants (see Saulny & Steinberg, 2011). 

Using the 1997 Standards and U.S. Census form as our models, the main objectives of 
this study were to determine how the question content and fonnat might affect the accuracy of 


4 



the response (i.e., the extent to which registrants understand and produce the response that the 
question is intended to elicit) and registrants’ comfort level with the question. Given the 
subjectivity and fluidity of ethnic and racial self-identification, our focus on accuracy consisted 
of simply asking participants whether they understood the questions about race and ethnicity. We 
also asked participants to indicate if these questions made them uncomfortable. Because this 
study is intended to inform ETS about how to best operationalize questions about race and 
ethnicity, we also inquired about how the questions should be changed if they were 
misunderstood or caused discomfort. 

Concerns About International Test-Takers 

In addition to examining race and ethnicity, we also looked at citizenship and country of 
birth. Issues of race and ethnicity are complicated by the fact that, over the past several years, 
about 25% of those who have registered for the GRE General Test reported being a citizen of a 
country other than the United States. Because the BIQ allows a registrant to report only one 
citizenship, it is possible that some respondents classified as non-U.S. citizens are actually dual 
U.S. citizens. Differences in how race and ethnicity are conceptualized and defined across 
nations (and sometimes within those nations) make the collection of racial and ethnic data more 
complex. The BIQ requests racial and ethnic information from U.S. citizens only. The number of 
non-U.S. citizens who go on to graduate and professional programs in the United States has been 
increasing markedly (Appel, 2011). Collecting racial and ethnic data for non-U.S. citizens might 
result in infonnation useful to those who wish to examine if DIF and DP occur across racial and 
ethnic groups outside of a traditional U.S. context. Non-U.S. educational institutions, where 
racial and ethnic equity typically uses a very different framework, might seek to use the GRE 
General Test for admissions and other purposes. For example, a Gennan university might 
primarily be concerned about whether the GRE Verbal Reasoning section functions similarly for 
ethnic Germans and Germans of recent Turkish descent (because Turkish immigrant families 
comprise a sizeable minority in Gennany and have experienced particular educational challenges 
there; see Mandell, 2008). The large number of non-U.S. citizens who take the GRE General 
Test also raises the question of whether differences in citizenship should be examined in the 
same way as differences in race and ethnicity. Sometimes citizenship serves as the basis for 
groups examined in DIF analyses when it represents variability in cultural norms for a 
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personality assessment or serves as a proxy for differences in language familiarity for a measure 
of nonverbal abilities (see, e.g., Huang, Church, & Katigbak, 1997). 

Our response to these issues of citizenship and country of birth was twofold. In addition 
to asking questions about race and ethnicity from a United States perspective, we developed and 
asked questions that we thought would capture racial and ethnic differences for a broad range of 
non-U.S. contexts. Much of the content of these questions came from the Organisation for 
Economic Co-operation and Development’s (OECD’s) 2006 Programme for International 
Student Assessment (PISA) Student Questionnaire and International Association for the 
Evaluation of Educational Achievement’s (IEA’s) 2007 Trends in International Mathematics and 
Science Study (TIMSS) Student Questionnaire. Both questionnaires inquire about country of 
birth for the registrants and their parents, how old the registrants were when they came to the 
country in which the test was being taken, and the language spoken at the registrants’ home (see 
Foy & Olsen, 2009; OECD, 2005). Furthermore, for the U.S.-style questions about race and 
ethnicity we examined citizenship and country of birth as individual difference variables within 
the larger context of race and ethnicity. We hoped that, among other things, this approach would 
clarify whether the U.S.-style questions about race and ethnicity were appropriate for those who 
are not U.S. citizens, were born outside of the United States, and/or are not pennanent U.S. 
residents. 


Methodology 

Sample 

Sampling for this study is a fundamental issue. It detennines the groups about which we 
can make inferences. A sample consisting largely of monoracial White U.S. citizens would 
provide very limited information to the authors and others concerned about the possibility that 
GRE items mismeasure examinees’ skills due to their race or ethnicity. We assumed that 
traditionally recognized ethnic and racial minorities that are underrepresented in quaternary 
education (i.e., the Hispanic and monoracial Black communities) as well as minorities not 
traditionally recognized in ethnic and racial analyses but who arguably should be (i.e., 
multiracial respondents and international students) would be most at risk of being misidcntificd 
if questions about race and ethnicity are poorly understood or cause discomfort. 

Misidentification, in an operational context, undermines the legitimacy of DIF and DP analyses. 
Although DIF and DP can negatively impact majorities well-represented in quaternary education, 
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researchers generally conduct DIF and DP analyses with the goal of revealing assessments and 
items that mismeasure the skills of underrepresented groups. By definition, underrepresented 
groups are often small in size (at least in a relative sense), so misidentification of their members 
can be especially harmful. Historically, underrepresented groups have been subject to overt 
discrimination as well as insensitive treatment. Therefore, it was especially important to try to 
obtain sample sizes large enough to detennine how best to appropriately elicit information from 
these groups in particular. 

The analyses in this study are based on responses from 720 individuals who responded to 
a final survey. In order to obtain a diverse sample while minimizing sampling error and cost, the 
original target sample consisted of 1,000 individuals who had registered to take the GRE General 
Test in the prior 3 years. For these registrants, we could obtain racial, ethnic, citizenship, 
residency, and contact information from the BIQ. In order to increase the likelihood that we 
would receive survey responses from registrants of a wide range of races/ethnicities, multiracial 
identities, and citizenships, we created specific numerical targets as part of a stratified sampling 
design (see Table 1). We initially based these targets on monoracial and citizenship frequency 
information that we obtained for respondents to the BIQ over the past 3 years. As described in 
greater detail below, we then adjusted this infonnation so that we could oversample certain 
groups whose opinions we believed to be especially important for purposes of this study but who 
might represent a very small percentage of GRE registrants (e.g., multiracial Black registrants). 
The stratifications included separate monoracial and multiracial categories for U.S. citizens who 
described themselves in the BIQ as American Indian/Alaska Native, Asian, Native Hawaiian or 
Pacific Islander, Black/African-American, Hispanic/Latino, or White. Registrants could choose 
only a single racial and ethnic group in the BIQ, but by using procedures described below we 
were able to preselect participants for the final survey based on their probability of describing 
themselves as monoracial or multiracial, if given the opportunity to do so. For the most part, 
available BIQ racial and ethnic infonnation was limited to U.S. citizens, because the BIQ’s 
question about race and ethnicity specifically requested a response from U.S. citizens only. 
Additional subgroups included citizens of nations in Africa, Asia, Australia/Oceania/Polynesia, 
Europe, and Caribbean/Latin America. In the BIQ, registrants could choose only a single 
citizenship. 
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Table 1 


Sampling Targets and Results 



Target n 

Resulting n 

U.S. citizens: 

750 

460 (388 U.S.-only + 72 dual U.S.) 

Multiracial, with self-identification as partly: 

Blaclc/African American 

113 

36 

Asian 

89 

15 

Native Hawaiian or Other Pacific Islander 

15 

3 

American Indian/Alaska Native 

112 

82 

White 

38 

101 

Hispanic/Latino 

188 

115 

Monoracial (or no indication of being multiracial): 

Black/African American 

94 

127 

Asian 

40 

56 

Native Hawaiian or Other Pacific Islander 

15 

0 

American Indian/Alaska Native 

24 

38 

White 

22 

88 

Citizens of countries in: 

250 

339 (245 single-country + 94 dual) 

Asia 

48 

107 [84] 

Europe 

47 

59 [39] 

Africa 

65 

83 [60] 

Caribbean/ Latin America 

65 

72 [48] 

Australia/Oceania/Polynesia 

25 

18 [14] 

Total 

1,000 



Note. Target n values represent independent persons. Unless otherwise indicated, resulting n 
values are not independent and therefore they sum to more than 720 (720 = the number of 
distinct respondents to the final survey) or to more than the totals in the rows headed U.S. 
citizens and Citizens of countries in. For example, a White Hispanic participant can be counted 
twice—once in the Hispanic/Latino row and then again in the monoracial White row. As another 
example, a U.S. citizen may be counted once in the multiracial Asian row and then again in the 
multiracial American Indian/Alaska Native row. For continent of citizenship (lower third of the 
table), a dual citizen is counted more than once for the resulting n values that are not in square 
brackets. Some dual citizens may be U.S. citizens. [ ] = totals for those with only one citizenship. 
Resulting n values for U.S. citizens are based on data from Format C of the final survey. 

Furthermore, this study was designed to oversample (a) Black participants, Hispanic 
participants, and Asian participants who were U.S. citizens and (b) citizens of African, 
Caribbean, and Latin American countries. It was unclear how to calibrate our targets for Native 
Hawaiian and other Pacific Islander participants who were U.S. citizens, given their tiny 
percentage of the test-taking population. To a lesser extent, the same issue existed for American 
Indian/Alaska Native participants. Over the past few decades in the United States, issues of 
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fairness and equal opportunity with regard to race and ethnicity have focused principally on 
Black and Hispanic U.S. citizens, but with some focus on Asian U.S. citizens as well (see, e.g., 
Zwick, 2002). Also, empirical research indicates that among the commonly compared racial and 
ethnic groups, African American and Hispanic test-takers attain the lowest mean group scores 
among takers of the GRE General Test while Asian test-takers attain the highest (particularly on 
the Quantitative Reasoning subtest; Gallagher, Bridgeman, & Cahalan, 2000). With regard to 
educational assessment, the need to address fairness issues (underprediction, overprediction, etc.) 
for these groups is therefore important. The study was also designed to oversample citizens of 
African nations under the assumption that most citizens of African countries would self-report as 
Black in response to a U.S.-style question about race. To a lesser extent, we had this expectation 
for oversampled citizens of Caribbean and Latin American countries as well. We also wanted to 
be able to compare responses from non-U.S. citizens who would self-report as Hispanic against 
responses from Hispanic U.S. citizens. Statistical analyses show that African-born, Caribbean- 
born, and Latin American-born Black test-takers in the United States on average attain higher 
levels of education than Black American test-takers whose families have resided in the United 
States for several generations (African-Bom U.S. Residents, 1996; Kent, 2007). For that reason, 
as well as due to the possibility that grouping Black non-U.S. citizens with Black U.S. citizens in 
the same fairness analysis might obscure differences between each of those subgroups and other 
racial groups, it would be helpful for future educational research (including DIF and DP studies) 
to also pay particular attention to Black test-takers who are not U.S. citizens or who were bom 
outside of the United States. 

Although the target sample consisted of 1,000 participants, there were only 720 
participants in the final sample because attempts to obtain all demographic targets became 
prohibitively difficult and expensive. Because the BIQ pennits the selection of only one choice 
among response options that treat Hispanic and racial categories separately, the BIQ cannot 
provide information about one’s identification with more than one race, one’s racial background 
if one indicated that one was Hispanic, or one’s Hispanic background if one indicated 
membership in a racial group. Furthennore, the BIQ does not permit indication of multiple 
citizenships, and it is unclear whether people with both a U.S. and other citizenship should be 
grouped with U.S. citizens, non-U.S. citizens, both, or neither for purposes of this study (or for 
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any fairness analyses). It seemed prudent to have the ability to categorize dual citizens as a 
distinct group. 

Given these challenges and the desire to recruit a certain number of people from different 
groups, we used an interim short survey to increase the prospects of achieving monoracial, 
multiracial, and Hispanic targets while accounting for those with multiple citizenships. We 
attempted to achieve our targets by first using BIQ demographic and contact infonnation to e- 
mail a short survey to GRE registrants most likely to lit those targets. In the short survey, we 
asked about race (including the possibility of self-identification with more than one race), 
possible Hispanic ethnicity, and citizenship. Recipients were told that if they responded to the 
short survey, then they might be selected to participate in the full study for compensation (a $25 
Visa gift card). When we received a completed survey that indicated that a person was likely to 
fit a demographic target, we then e-mailed a full survey to that individual. We informed 
respondents who, based on their responses to the short survey, were unlikely to satisfy a 
demographic target that they would not be invited to participate in the full study. Recruiting 
certain demographic backgrounds (e.g., multiracial Black and multiracial Asian participants) 
eventually became so labor-intensive, time-consuming, and thus potentially expensive that we 
had to cease recruitment even though recruitment targets were not fully met. Ultimately, out of 
the 10,507 test-takers to whom we e-mailed short surveys, 1,362 responded (a 13% response 
rate), and we selected for participation in the final survey 720 out of those 1,362 respondents (a 
53% selection ratio). 

The resulting demographics appear in Table 1. In relation to respective target numbers in 
the table, White U.S. citizens were especially oversampled, and multiracial Black and multiracial 
Asian U.S. citizens were notably undersampled. When a response to the question about race and 
ethnicity in the final survey did not match a response to the question about race and ethnicity in 
the interim short survey, it typically resulted in the count of another monoracial or multiracial 
White participant. Because the substantial majority of U.S. citizens who take the GRE General 
Test report that they are White, this result is not surprising. In general, it was extremely difficult 
to recruit Native Hawaiian and other Pacific Islander test-takers (all U.S. citizens). Citizens of 
Asian countries were oversampled. When a response to the citizenship questions in the final 
survey did not match a response to the citizenship question in the interim short survey, it 
typically resulted in the count of another citizen of an Asian country. Ultimately, our recruitment 
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of participants resulted in small sample sizes for some of the subgroups analyzed in this study 
(e.g., multiracial Asian participants). Nevertheless, we believe that it is a lesser offense to 
analyze small samples for the subgroups that should be the focus of this study than to lose 
completely the ability to infer anything about these subgroups as a result of aggregating them 
into larger groups (e.g., all Asian participants). 1 

Measures 

Interim Short Survey 

The interim short survey, which was sent as an e-mail attachment, requested that the 
recipient provide basic contact and citizenship information and check Yes or No for whether the 
respondent sometimes thought of herself/himself as belonging to more than one racial and/or 
ethnic group. For those who responded Yes, the survey asked the respondent to write in the racial 
and ethnic groups with which the respondent identified. 

Final Survey 

Recipients of the final survey received an e-mail message with a hyperlink that took them 
to the final survey. The survey had six versions and each participant was randomly assigned to 
one of them. The versions were substantively identical. (Figures 1-16 comprise one full version 
of the survey.) They differed only in the presentation order of three question formats for the 
U.S.-style questions about race and ethnicity (see formats A, B, and C in Figures 13-15 of the 
full survey). There were six survey versions administered (sequences ABC, ACB, BAC, BCA, 
CAB, and CBA) to eliminate ordering effects that might have affected differences in response 
patterns. 

Aside from an initial informed consent section and a closing section that sought personal 
contact information from each respondent, the final survey consisted of approximately 31 items. 
This is an approximation, because the final survey was in an online format that routed 
respondents to items based on responses to previous items. The survey consisted of two halves, 
with the first half (Figures 1-11 and part of Figure 16) created with an international group of 
test-takers in mind and the second half (Figures 12-16) created with U.S. test-takers in mind. We 
placed the international-style questions first, because we felt that the international-style questions 
would make equal sense to U.S. citizens and non-U.S. citizens but that the U.S.-style questions 
might make less sense to non-U.S. citizens. By placing the U.S.-style questions last, we 
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eliminated the risk that they would negatively affect the responses of non-U. S. citizens to the 
international-style questions. The international-style questions (first half) consisted of about 14 
items, and the U.S.-style questions (second half) consisted of 17 items. Using drop-down menus, 
radio buttons, check boxes, and text boxes, the international-style questions asked about 
citizenship status, residency, birth country, birth country of parents/guardians, whether or not the 
respondent resided in an English-speaking country for at least one half of the respondent’s life, 
up to two languages spoken primarily at home, and English proficiency level. Using similar 
formats, the U.S.-style questions asked about Hispanic origin, race, and ethnicity; the extent to 
which respondents understood, were comfortable with, and wanted to eliminate or change the 
three question formats that were posed; and which of the three question formats respondents 
preferred and why. 

As discussed earlier, much of the content was based on the OECD’s 2006 PISA Student 
Questionnaire and IEA’s 2007 TIMSS Student Questionnaire. We allowed respondents to report 
more than one citizenship. Under the assumption that citizenship is a reasonable measure of 
culture, we were concerned that people with a U.S. citizenship and another citizenship might 
typically think about race and ethnicity very differently than people with just U.S. citizenship or 
just non-U.S. citizenship. Given uncertainties about the relevance of certain questions about race 
and ethnicity and frameworks for people of different cultures, this citizenship information was 
used for several analyses. 

Because the BIQ seeks parental information and the OECD’s 2006 PISA Student 
Questionnaire and IEA’s 2007 TIMSS Student Questionnaire both inquire about the country of 
birth for the test-taker and the test-taker’s parents, we included similar questions (see Figures 6 
and 8). Given the complexity of the modem family structure in the Western world and our 
particular uncertainty about who generally raises a child in non-Western cultures, we thought it 
prudent to ask about guardians and caregivers, in addition to parents. We also did not want to 
assume that the appropriate number of parents, guardians, or caregivers for a respondent was 
two, so we allowed the respondent to provide information for one to five of them (see Figures 7— 
8). We chose five as our upper bound, given the length of the survey and our assumption that few 
if any participants would choose more than five parents/guardians/caregivers. 

Furthermore, the survey requested information about language proficiency. The BIQ 
already asks whether the registrant’s English communication is at least as proficient as the 
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registrant’s communication in another language. In addition, PISA and TIMSS Student 
Questionnaires ask for the language spoken at the test-taker’s home. We added similar language 
proficiency content here not simply in the belief that information about language proficiency has 
its own merits, but largely with the idea that it can identify important racial or ethnic background 
infonnation about a registrant. For example, it can help distinguish a Canadian citizen whose 
primary language is French and who—when compared to a Canadian citizen whose primary 
language is English—may be at a disadvantage in terms of performance on the GRE Quantitative 
Reasoning section due to construct-irrelevant differences in English reading ability. Without 
language information, one might conclude that these test-takers are both monoracial White and 
non-Hispanic Canadian citizens who should always be grouped together for purposes of 
conducting DIF or DP analyses. 

As part of this focus on English proficiency, we incorporated the BIQ language 
proficiency question into the survey but with its response options broken out into more specific 
aspects of English communication ability (see Figure 11). The BIQ asks, “Do you communicate 
better (or as well) in English than in any other language?” However, there are different aspects of 
communication in English, and there can be major discrepancies in a person’s strengths across 
these different aspects. For example, one may speak English well but be unable to read it. In 
addition, the relevance of each aspect of English communication to performance on an ETS 
assessment may vary considerably. English reading ability is the aspect of English 
communication primarily relevant for success on the GRE Verbal Reasoning and Quantitative 
Reasoning subtests, whereas English writing ability is the aspect primarily relevant for success 
on the GRE Analytical Writing subtest. The separation of English proficiency into separate 
competencies would, therefore, allow ETS to collect information for a relevant English 
competency rather than for an irrelevant one. 
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GRE Test Taker Survey (123) 


CITIZENSHIP STATUS 

Country of citizenship? (NOTE: In a later question, you may choose a second country) 


Select One 

If you selected "None of the above”, please provide below your country of citizenship or reason 
for not choosing from the drop-down list of countries. 



Figure 1. Citizenship status. 



Listening. Learning. Leading.’ 



GRE Test Taker Survey (123) 


Do you possess citizenship from a another country, too? 

©Yes 
C 1 No 


Figure 2. Do you possess citizenship from another country? 
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Listening. Learning. Leading.' 



GRE Test Taker Survey (123) 


From what other country do you possess citizenship? 


Select One ▼ 

If you selected "None of the above", please provide below your other country of citizenship or 
reason for not choosing from the drop-down list of countries. 



Figure 3. From what other country do you possess citizenship? 



'ETS 


Listening. Learning. Leading.’ 



GRE Test Taker Survey (123) 


Are you a nonresident alien or a permanent resident in the United States? 

© Yes 
©No 


Figure 4. Are you a nonresident alien or a permanent resident in the United States? 
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Listening. Learning. Leading.' 



GRE Test Taker Survey (123) 


If you are a United States citizen, resident alien in the United States, or permanent resident in 
the United States, what U.S. state or territory do you consider to be your permanent U.S. 
residence? 


Select One 


Figure 5. If you are a United States citizen, resident alien in the United States, or 
permanent resident in the United states, what U.S. state or territory do you consider to be 
your permanent U.S. residence? 



Listening. Learning. Leading.’ 


GRE Test Taker Survey (123) 


In what country were you born? 


Select One 

If you selected "None of the above", please provide below your country of birth or reason for not 
choosing from the drop-down list of countries. 



Figure 6. In what country were you born? 
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Listening. Learning. Leading.' 


GRE Test Taker Survey (123) 


How many parents, guardians, or other caregivers raised you? You may choose up to 5. 


Select One ▼ 


Figure 7. How many parents, guardians, or other caregivers raised you? 
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GRE Test Taker Survey (123) 


Please describe parent/guardian/caregiver 1 who raised you: 

Select One ▼ 

In what country was your parent/guardian/caregiver 1 who raised you born? 

Select One ▼ 

If you selected "None of the above", please provide below the country of birth of 
parent/guardian/caregiver 1 or reason for not choosing from the drop-down list of countries. 


Figure 8. Please describe Parent/Guardian/Caregiver 1 who raised you. 


17 









Listening. Learning. Leading.' 



GRE Test Taker Survey (123) 


Have you resided in an English-speaking country for at least one-half of your life? 

© Yes 
©No 


Figure 9. Have you resided in an English-speaking country for at least one half of your life? 



GRE Test Taker Survey (123) 


What languages were primarily spoken in the home in which you were raised? You may choose 
up to 2 languages. 

NOTE: If only one language was primarily spoken, then after chosing language #1, please click on 
"Next Page" below. 

Language #1 

Select One - 

If a language does not appear in the drop-down menu list, please provide it below: 


Language #2 

Select One ▼ 

If a language does not appear in the drop-down menu list, please provide it below: 


Figure 10. What languages were primarily spoken in the home in which you were raised? 
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GRE Test Taker Survey (123) 


Do you communicate better (or as well as) in English than in any other language? (Select all that 
apply.) 

□ Yes, for understanding others' spoken words (listening comprehension) 

□ Yes, for speaking to others (spoken communication) 

□ Yes, for reading (reading comprehension) 

□ Yes, for writing to others (written communication) 

1J No, not at all 


Figure 11. Do you communicate better (or as well as) in English than in any other 
language? 

The second half of the survey (see Figures 12-16) focuses on U.S.-style categorizations 
of race and ethnicity. There were three formats for the question about race and ethnicity: Format 
A, Format B, and Format C. Fonnat A (Figure 13) is identical to the current BIQ question about 
race and ethnicity in terms of substance and format. It represents ETS’s potential decision not to 
make any changes to the BIQ question about race and ethnicity. Also, it permits a 
straightforward analysis of the stability of racial and ethnic self-identification (a comparison 
between BIQ responses and final survey responses). Granted, self-reported race/ethnicity is 
subjective and fluid. Nevertheless, if stability is too low, then it may be impossible to generalize 
many of the results from this study. Format C (Figure 15) represents the most pluralistic 
questions about race and ethnicity of the three formats and is virtually identical to the questions 
about race and ethnicity of the 2010 U.S. Census. We believe that Format C satisfies (and 
exceeds) the OMB’s 1997 Standards as implemented by the DOE. Among other things, the DOE 
requires the following for data collection purposes (Final Guidance on Maintaining, Collecting, 
and Reporting Racial and Ethnic Data to the U.S. Department of Education, 2007): 
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1. the use of the following racial categories that, at minimum (except for No Child Left 
Behind), should be (a) American Indian or Alaska Native, (b) Asian, (c) Black or 
African American, (d) Native Hawaiian or Other Pacific Islander, and (e) White; 

2. the use of a two-question format with a race question pennitting the reporting of more 
than one race from the five required groups; and 

3. that if there is a question asking whether the person is Hispanic, it precedes the race 
question. 



Listening. Learning. Leading.’ 


GRE Test Taker Survey (123) 


On each of the next three pages, you will see a different format for questions about race and 
ethnicity. Please answer the questions for all three formats. After you answer all of these 
questions, you will have an opportunity to tell us which format, if any, you prefer and why. 


Figure 12. Introduction to questions on race and ethnicity. 
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GRE Test Taker Survey (123) 


IJ.IJi'iWJJ 


How do you describe yourself? (select one) 

O American Indian or Alaska Native 
Asian or Asian American 
£' Black or African American 
<£j Mexican, Mexican American or Chicano 
O Native Hawaiian or Other Pacific islander 
© Puerto Rican 

C' Other Hispanic, Latino, or Latin American 
C« White (non-Hispanic) 

© Other 

If you did not adequately understand the preceding question or its response choices/ please tell us 7 and explain why below: 



If the preceding question or its response choices made you feel uncomfortable in any way 7 please tell us, and explain why below: 



If you think that the preceding question or its response choices should be eliminated or changed in any way, please tell us below. Please specify 
any changes that you think should be made: 



Figure 13. Format A. 
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GRE Test Taker Survey (123) 



Are you of Hispanic Latino, or Spanish origin? 

O No, not of Hispanic, Latino, or Spanish origin 
C'Ves, Mexican, Mexican-American, Chicano 
C'Ves, Puerto Rican 
C'Ves, Cuban 

C'Ves, another Hispanic, Latino, or Spanish origin (Type in origin, for example, Argentinian, Colombian, Dominican, 
Nicaraguan, Salvadoran, Spaniard, and soon) >k _ 


How do you describe yourself? 

O American Indian or Alaska C' Native Hawaiian or Other Pacific Islander 


Native 

C Asian or Asian American 
C Black or African American 


C' White 

O Other - Type in race below vk 


If you did not adequately understand the preceding questions or their response choices, please tell us, and explain why below: 


If the preceding questions or their response choices made you feel uncomfortable in any way, please tell us, and explain why below: 


If you think that the preceding questions or their response choices should be eliminated or changed in any way, please tell us below. Please 
specify any changes that you think should be made: 


Figure 14. Format B. 
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GRE Test Taker Survey (123) 


Format C 


Please answer BOTH questions about Hispanic origin and race. For Format C, Hispanic origins are not races. 

Are you of Hispanic* Latino, or Spanish origin? 

No, not of Hispanic. Lafcno. or Spanish origin 
'. Yes, Mexican, Mexcar-Amencar, Chicano 
Yes, Puerto Rican 
Yes. Cuban 

•_ Yes, another Hisparic. Latino, or Spanish oogai - Type tn origin, For example, Argentinian, Colombian, Dominican, 
Nicaraguan, Salvadoran, Spaniard, and so on i 


What is your race (and ancestry)? (Select all that apply.) 

□ White 

□ Stack or African American 

□ American Indian or Alaska Native- Type tn name of enrolled or prtnerpoi otbe i 


□ Asian Indian 

□ Chinese 


□ Japarese 

□ Korean 


□ Native Hawaiian 

□ Guamanian or Chamorro 


□ FSpiro Vietnamese 

tH Other Asian - Type in race, far 
example. Hmong. iaeOon.Thet. Poboeont, 

Canreadten, and wo on I 


i~l Samoan 

□ Cther Pacific Islander - Type tn race, for 
example, njten. Tongen, end as on J 


□ Some other race - Type tn race I 


If you did not adequately understand the preceding questions or their response choices, please tell us, and explain why below; 


If the preceding questions or their response choices made you feel uncomfortable in any way, please tell us, and explain why below; 



If you think that the preceding questions or theirs response choices should be eliminated or changed in any way, please tell us below. Please specify any 
changes that you think should be made; 



Figure 15. Format C, 
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Listening. Learning. Leading .* 



GRE Test Taker Survey (123) 


Which format (Format A, Format B, or Format C) do you prefer? 

If you would like to review the three formats, please click on the following link: Formats 

© Format A 
© Format B 
© Format C 
© None of the above 

Please explain below why you prefer that particular format. If you did not like any of the formats, 
please explain why, and feel free to suggest a different format below. 



We also would appreciate below your general comments about this survey and the other 
questions and response choices in it. 

If you would like to review the other survey questions and response choices, please click on the 
following link: Survey questions and response choices 



Figure 16. Which format do you prefer and why? 

The DOE requires, for data reporting purposes (possibly limited to when reporting is 
made to DOE), aggregated data in the following seven categories: (a) Hispanic/Latino of any 
race and for test-takers who are non-Hispanic/Latino only, (b) American Indian or Alaska 
Native, (c) Asian, (d) Black or African American, (e) Native Hawaiian or Other Pacific Islander, 
(f) White, and (g) two or more races (Final Guidance on Maintaining, Collecting, and Reporting 
Racial and Ethnic Data to the U.S. Department of Education, 2007). The implication is that the 
data collection process can break each of the first six groups into finer distinctions. In addition, 
there are guidelines for when third-party identification of race and ethnicity is permitted or 
required, maintaining and bridging to records with racial and ethnic infonnation in an old format, 
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and there are special rules for No Child Left Behind. ETS should refer to the DOE requirements 
for further infonnation. It is possible that the 1997 Standards impose additional data collection 
and reporting requirements, and different DOE and OMB requirements may apply to different 
ETS programs. 

We based Format C (Figure 15) of the final survey on the 2010 Census questions about 
race and ethnicity in part because a tremendous amount of research conducted by the U.S. 

Census Bureau has served as the main empirical foundation for the 1997 Standards (Revisions to 
the Standards for the Classification of Federal Data on Race and Ethnicity, 1997). The 1997 
Standards established the parameters for the questions about race and ethnicity in the 2000 U.S. 
Census questionnaire and the surveys, forms, and records used by the U.S. federal government’s 
departments and agencies as of January 2003. For the 2010 U.S. Census, the U.S. Census Bureau 
conducted additional research (e.g., the 2003 and 2005 National Census Tests) to collect data in 
an effort to determine if and how questions about race and ethnicity from the 2000 Census 
questionnaire should be amended. We believed it sound and prudent to offer Fonnat C for 
feedback purposes whether or not it should be used in an operational context. 

Format B (Figure 14) is a hybrid of Format A and Fonnat C. It separates the one-part 
question of Format A into two parts, with the first question about Hispanic origin and the second 
question about race. However, it does not permit the respondent to choose more than one race. 
Therefore, it does not fully comply with the DOE’s implementation of the 1997 Standards. 
Regardless, we felt that, for the purpose of this empirical study, the responses to Fonnat B would 
provide useful context for Fonnats A and C. 

Analysis, Results, and Discussion 
Stability of Racial and Ethnic Self-Identification 

Racial self-identification without a clear set of common rules is at least somewhat 
subjective and fluid. Nevertheless, sufficient stability in self-identification is required for one to 
be able to make predictions based on it. In other words, validity requires reliability. We were 
able to compare racial and ethnic self-identification on the BIQ to racial and ethnic self- 
identification on the final survey (Format A) where response options for Hispanic background 
and response options for race appear together in a single question, and a respondent can select 
only one response option. Racial and ethnic self-identification when the respondent has a single 
response option is relatively stable (see Table 2). Values for % within BIQ indicate the 
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percentage of respondents to the BIQ who chose the same racial and ethnic group later in the 
final survey (Fonnat A). Values for % within Final Survey Format A indicate the percentage of 
respondents to the final survey who chose the same racial and ethnic group earlier in the BIQ. At 
first glance, it would appear that all or almost all of those who had self-reported on the final 
survey (Fonnat A) as Puerto Rican, Mexican, Mexican American, or Chicano self-reported as 
Other Hispanic or Latin American on the BIQ (the adjacent 100% and 78% values in Table 2 
where the Other Hispanic or Latin American category for the BIQ intersects with the Puerto 
Rican and Mexican, Mexican American, or Chicano categories for the final survey). Similarly, it 
would seem that all of those who self-reported on the final survey (Fonnat A) as Native 
Hawaiian or Pacific Islander self-reported as Asian, Asian American, or Pacific Islander on the 
BIQ (the 100% value in Table 2 where the Asian, Asian American, or Pacific Islander category 
for the BIQ intersects with the Native Hawaiian or Other Pacific Islander category for the final 
survey). However, these results may not be as they seem. The BIQ provides separate response 
options for Puerto Rican, Mexican, Mexican American, or Chicano and for Native Hawaiian or 
Other Pacific Islander. Regardless of how the racial and ethnic infonnation is collected, it 
nevertheless was internally reported to the authors in an aggregated fashion such that (a) all 
Hispanic participants were grouped together, and (b) Native Hawaiian and Pacific Islander 
participants were grouped with Asian test-takers. The DOE requires that Native Hawaiian or 
Other Pacific Islander remains a distinct category for data collection and reporting to the DOE 
(Final Guidance on Maintaining, Collecting, and Reporting Racial and Ethnic Data to the U.S. 
Department of Education, 2007). Finally, those who simply reported as Other in the final survey 
were split fairly evenly among BIQ groups (the 37%, 23%, and 23% values in Table 2 where 
American Indian, Alaskan Native, or Other Native American; Black/African American; and 
Other Hispanic or Latin American categories for the BIQ intersect with the Other category for 
the final survey). Aggregation issues aside, there is enough stability in racial and ethnic self- 
reporting over time to justify use of the final survey to investigate racial and ethnic self-reporting 
further. 
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Table 2 

Consistency of Single-Choice Racial and Ethnic Self-Identification Over Time (GRE Background Information Questionnaire 
Versus Final Survey Format A) 


Final Survey Format A 


Background Information Questionnaire (BIQ) 

American 
Indian or 
Alaska 
Native 

Asian or 
Asian 
American 

Black or 
African 
American 

Other 
Hispanic, 
Latino, or 
Latin 
American 

White 

(Non- 

Hispanic) 

Puerto 

Rican 

Mexican, 
Mexican 
American, or 
Chicano 

Native 
Hawaiian or 
Other Pacific 
Islander 

Other 

Total 

American Indian, 

n 

82 

0 

0 

0 

12 

0 

0 

0 

11 

105 

Alaskan Native, 

% within BIQ 

78% a 

0% 

0% 

0% 

11% 

0% 

0% 

0% 

10% 

100% 

or Other Native 
American Group 

% within Final Survey Format A 

100% b 

0% 

0% 

0% 

27% 

0% 

0% 

0% 

37% e 

23% 

Asian, Asian 

n 

0 

58 

0 

0 

0 

0 

1 

2 

3 

64 

American, or 

% within BIQ 

0% 

91% a 

0% 

0% 

0% 

0% 

2% 

3% 

5% 

100% 

Pacific Islander 

% within Final Survey Format A 

0% 

100% b 

0% 

0% 

0% 

0% 

11% 

100% d 

10% 

14% 

Black/African 

n 

0 

0 

145 

0 

0 

0 

0 

0 

7 

152 

American 

% within BIQ 

0% 

0% 

95% a 

0% 

0% 

0% 

0% 

0% 

5% 

100% 


% within Final Survey Format A 

0% 

0% 

100% b 

0% 

0% 

0% 

0% 

0% 

23% e 

33% 

Other Hispanic or n 

0 

0 

0 

89 

1 

1 

7 

0 

7 

105 

Latin American 

% within BIQ 

0% 

0% 

0% 

85% a 

1% 

1% 

7% 

0% 

7% 

100% 


% within Final Survey Format A 

0% 

0% 

0% 

100% b 

2% 

100% c 

78% c 

0% 

23% e 

23% 

White 

n 

0 

0 

0 

0 

29 

0 

1 

0 

2 

32 

(Non-Hispanic) 

% within BIQ 

0% 

0% 

0% 

0% 

91% a 

0% 

3% 

0% 

6% 

100% 


% within Final Survey Format A 

0% 

0% 

0% 

0% 

66% b 

0% 

11% 

0% 

7% 

7% 

Nothing reported 

n 

0 

0 

0 

0 

2 

0 

0 

0 

0 

2 


% within BIQ 

0% 

0% 

0% 

0% 

100% 

0% 

0% 

0% 

0% 

100% 


% within Final Survey Format A 

0% 

0% 

0% 

0% 

5% 

0% 

0% 

0% 

0% 

0% 

Total 

N 

82 

58 

145 

89 

44 

1 

9 

2 

30 

460 


% within BIQ 

18% 

13% 

32% 

19% 

10% 

0% 

2% 

0% 

7% 

100% 


% within Final Survey Fonnat A 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 


a These values (% within BIQ) indicate the percentage of respondents to the BIQ who chose the same racial or ethnic group later in the final 
survey (Format A). b These values (% within Final Survey Fonnat A) indicate the percentage of respondents to the final survey who chose 
the same racial or ethnic group earlier in the BIQ. C lndicates consistency in reporting for Hispanic and Latin American groups when the 
BIQ subcategories for Hispanic and Latin American groups are combined, indicates consistency in reporting for Native Hawaiian and 
Other Pacific Islander groups when the BIQ category for Native Hawaiians and Other Pacific Islanders is combined with the BIQ category 
for Asians and Asian Americans, indicates how those who reported as Other in the Final Survey (Format A) primarily reported in the BIQ. 



The U.S.-Style Questions About Race and Ethnicity 

Preferences. In general, participants preferred the question format that they felt was most 
relevant to their racial and ethnic backgrounds. Tables 3 through 15 generally show that 
regardless of background, respondents preferred Format A or Format C (i.e., almost never 
Format B). Table 3 shows that respondents were fairly evenly split between preferring Formats 
A and C, with Format A being only slightly preferred. We expected that Format A, with its one- 
question design that permits only one response to items that treat Hispanic origin as equivalent to 
race, would be most preferred by monoracial respondents, non-Hispanic respondents, and 
persons who do not necessarily relate to or desire a question format based on U.S. standards of 
race and ethnicity (e.g., non-U.S. citizens). One would expect Format C’s great pluralism to 
appeal to Hispanic, multiracial, Asian, and Native American respondents. Format C recognizes 
Hispanic origin separately from race; it permits multiracial respondents to describe themselves as 
such; it allows Native American respondents to report their tribes; and it provides several explicit 
subcategories for Asian respondents (Chinese, Korean, etc.). 

Table 3 


All Respondents 


Format A 

Format B 

Format C 

None of the above 

38.3 % a n = 276 

15.7% n = 113 

36.7% n = 264 

9.3% n = 67 


“Preferred format is in boldface. 


We believed that those who are not personally familiar with the way that race and 
ethnicity are perceived in the United States would generally prefer Format A because, as the 
most streamlined of the fonnats, it is the option to which such registrants will best relate. We 
will refer to those having an understanding of and preference for racial and ethnic categories that 
are shaped primarily by U.S. culture as being U.S.-acculturated and everyone else as being not 
U.S.-acculturated. 2 We also believed that only U.S.-acculturated registrants would, on the whole, 
prefer Format C, because its complexity is irrelevant and perhaps confusing in general to 
registrants who are not U.S.-acculturated. Format C is based on U.S. Census Bureau research 
that determined it to be the most appropriate fonnat for the U.S. population. Nevertheless, we 
believed that having or acquiring U.S. citizenship (especially as one’s only citizenship) and 
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being born in the United States would typically indicate socialization in favor of a more 
pluralistic question in the form of Format C. The data in Tables 4 and 5 support our beliefs about 
the effects of socialization on a preference for more pluralistic questions. Having dual citizenship 
(U.S. + other citizenship) may indicate acculturation partially or predominantly to non-U.S. 
nonns, which might explain why U.S. citizens with dual citizenship do not prefer Format C over 
Format A (see Table 4). However, Fonnat C is preferred by U.S. citizens with dual citizenship 
who were bom in the United States as well as by Americans with sole U.S. citizenship who were 
bom outside of the United States (see Table 5). 


Table 4 

By Citizenship/Residency Status 

None of the 


Citizenship/residency status Format A Format B Format C above 


U.S. citizenship only 

36% (77 = 

140) 

15% 

(77 = 

57) 

41% 

“ (77 

= 160) 

II 

N® 

© X 

oo 

31) 

U.S. + other citizenship 

38% a 

(77 = 

= 27) 

19% 

(77 = 

14) 

36% 

(77 = 

26) 

7% (77 = 

5) 

Other citizenship only 












U.S. permanent resident /alien 

37% a 

(77 = 

= 21) 

18% 

(77 = 

10) 

33% 

(77 = 

19) 

12% (77 

= 7) 

Not U.S. permanent resident /alien 

43 % a 

(77 = 

= 87) 

16% 

(77 = 

32) 

29% 

(77 = 

59) 

12% (77 

= 24) 

Total (across citizenship/residency status) 

00 

6 S 

(N 

= 275) 

i 16% 

(N= 

= 113) 

37% 

(N= 

= 264) 

9% (N = 

= 67) 


Preferred format is in boldface. 


Table 5 


By Birthplace and Citizenship 


Citizenship/residency status 

Format A 

Format B 

Format C 

None of the 
above 

Born in the U.S. (native-born U.S. citizens) 












U.S. citizenship only 

36% 

(77 = 

124) 

15% 

(77 = 

53) 

41% 

(77 

= 143) 

8% (77 = 

27) 

U.S. + other citizenship 

39% 

(77 = 

11) 

14% 

(77 = 

4) 

46% 

(77 

= 13) 

0% (77 = 

0) 

Born outside of the U.S. 












U.S. citizenship only 

39% 

(77 = 

16) 

10% 

(77 = 

4) 

41% 

(77 

= 17) 

10% (77 = 

= 4) 

U.S. + other citizenship 

36% 

(77 

= 16) 

23% 

(77 = 

10) 

30% 

(77 = 

13) 

11% (77 = 

= 5) 

Other citizenship only 

42% 

> 

= 109) 

16% 

(77 = 

42) 

30% 

(77 = 

78) 

12% (77 = 

= 31) 

Subtotal for born outside of U.S. 

41% 

a (N 

= 141) 

16% 

(N= 

= 56) 

31% 

(N= 

= 108) 

12% (N 

= 40) 


“Preferred format is in boldface. 


In general, registrants who are not U.S.-acculturated prefer simpler Fonnat A more than 
other registrants do. In general, the pluralistic Format C will be preferred more by U.S.- 
acculturated registrants than by those who are not U.S.-acculturated, but there are important 
differences among U.S.-acculturated subgroups: With regard to question preference, a U.S.- 


29 





acculturated registrant’s mono/multiraciality appears to supersede the effects of being U.S.- 
acculturated. Monoracial U.S. citizens as well as monoracial non-U.S. citizens prefer format A 
(see Table 7). Moreover, a registrant who is a multiracial U.S. citizen is more likely to prefer 
Format C regardless of whether she or he is a dual citizen (see Table 7). Non-U. S. citizens prefer 
Fonnat A unless they are multiracial pennanent residents of the United States, in which case they 
prefer Fonnat C (see Table 7). Table 8 illustrates how Table 5 is not a complete story—unlike 
their multiracial counterparts, monoracial registrants bom in the United States actually prefer 
Fonnat A over Fonnat C. Also, multiracial registrants bom outside of the United States prefer 
Fonnat C to Fonnat A (see Table 8). In addition, the trend for U.S. citizens to favor Fonnat C 
does not hold for Black respondents or the majority of Hispanic respondents (see Table 6). 
However, that appears to be due to the fact that most of them identify as being monoracial (and 
thus prefer Format A). When one differentiates Black respondents and Hispanic respondents by 
mono/multiraciality, one observes that multiracial Black and multiracial Hispanic U.S. citizens 
prefer Fonnat C (see Tables 9 and 12). In fact, monoracial White U.S. citizens prefer Format 
A—it is the strong preference of multiracial White U.S. citizens for Format C that makes it seem 
that White U.S. citizens in general prefer Format C (see Table 11). 


Table 6 

U.S. Citizenship Only: Preferences by Race/Ethnicity (When Only One Race/Ethnicity Can Be 


Chosen) 


Race/ethnicity 

Format A 

Format B 

Format C 

None of the above 

American Indian or Alaska 

21.3% (« = 17) 

6.3% (n = 5) 

65.0% a (« = 52) 

7.5% (;n = 6) 

Native 

Asian or Asian American 

38.3% (n = 18) 

8.5% (n = 4) 

48.9% a (« = 23) 

4.3% (n = 2) 

Black or African American 

46.0% a (n = 57) 

20.2% (n = 25) 

27.4% (n = 34) 

6.5% (n = 8) 

Native Hawaiian or Other 

0.0% (n = 0) 

0.0% (n = 0) 

50.0% a (« = 1) 

50.0% a (« = 1) 

Pacific Islander 

White (non-Hispanic) 

41.9% (n = 13) 

6.5% (« = 2) 

51.6% a (« = 16) 

0.0% (n = 0) 

Mexican, Mexican American, or 

25.0% (n = 2) 

37.5% a (n = 3) 

12.5% (n = 1) 

25.0% (ns2) 

Chicano 

Puerto Rican 

0.0% (« = 0) 

0.0% (n = 0) 

100.0% a (n = 1) 

0.0% (n = 0) 

Other Hispanic, Latino, or Latin 

42.3 % a 0? = 30) 

21.1% (n = 15) 

25.4% (n = 18) 

11.3% (n« 8) 

American 

Other 

12.5% (n = 3) 

12.5% (n = 3) 

65.0% a (« = 14) 

16.7% (n= 4) 

Total 

36.1 %(N= 140) 

15.0% (N= 57) 

41.2 % a (A = 160) 

8.0% (A =31) 


“Preferred format is in boldface. 
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Table 7 

By Citizenship/Residency Status and Mono/Multiracial Self-Identification 


Citizenship/residency status and ethnicity/race Format A _ Format B _ Format C None of the above 

U.S. citizenship only 
Monoracial 
Multiracial 

U.S. + other citizenship 
Monoracial 
Multiracial 

Other citizenship only 
Monoracial 
Multiracial 

Non-U.S. permanent resident/alien 
Monoracial 
Multiracial 

U.S. permanent resident/alien 
Monoracial 
Multiracial 

“Preferred format is in boldface. 


Table 8 

By Birthplace and Mono/Multiracial Self-Identification 

Citizenship status and ethnicity/race Format A Format B Format C None of the above 

Born in the U.S. 

Monoracial 
Multiracial 

Born outside of the U.S 
Monoracial 
Multiracial 

“Preferred format is in boldface. 


Table 9 


By Hispanic/Non-fflspanic Self-Identification and Citizenship Status 


Citizenship status and ethnicity/race 

Format A 

Format B 

Format C 

None of the above 

All respondents (ignoring citizenship) 

Flispanic 

40.4% a (N= 64) 

21.7% (A =36) 

30.0% (A =47) 

13.4% (A = 19) 

Non-FIispanic 

38.3% (A =206) 

14.0% (A= 74) 

40.8% a (A= 214) 

10.5% (A =46) 

U.S. citizenship only 

Flispanic 

36.6% a (77 = 34) 

21.5% (77 = 20) 

26.9% (77 = 25) 

15.1% (77 = 14) 

Non-FIispanic 

36.1% (77 = 105) 

12.4% (77 = 36) 

46.0% a (77 = 134) 

5.5% (77 = 16) 

U.S. + other citizenship 

Flispanic 

24.0% (77 = 6) 

20.0% (77 = 5) 

44.0% a (77 = 11) 

12.0% (77 = 3) 

Non-FIispanic 

42.2% a (77 = 19) 

20.0% (77 = 9) 

33.3% ( 77 = 15) 

4.4% (77 = 2) 

Other citizenship only 

Flispanic 

50.0% a (77 = 24) 

22.9% ( 77 = 11) 

22.9% ( 77 = 11) 

4.2% (77 = 2) 

Non-FIispanic 

40.2% a (77 = 82) 

14.2% (77 = 29) 

31.9% (77 = 65) 

13.7% (77 =28) 


“Preferred format is in boldface. 


42.6% a (« =110) 
19.1% {n = 21) 


18.2% (77 = 47) 
7.3% (77 = 8) 


32.2% (77 = 83) 
66.4% a (77 = 73) 


7.0% (77 = 18) 
7.3% (77 = 8) 


43.1% a (77 = 125) 
30.8% (77 = 12) 


16.9% (77 = 49) 
15.4% (77 = 6 ) 


31.0% (77 = 90) 
35.9% a (77 = 14) 


9.0% (77 = 26) 
17.9% (77 = 7) 


42.6% a (77 = 113) 17.7% (77 = 47) 32.5% (77 = 86) 7.2% (77 =19) 

19.3% (77 = 22) 07.0% (77 = 8) 64.9% a (77 = 74) 8.8% (77 = 10) 


41.9% a (77 
12.5% (77 = 


= 26) 

1 ) 


22.6% ( 77 ; 
00.0% (77 : 


14) 

0 ) 


30.6% (77 = 19) 
75.0% a (77 = 6) 


4.8% (77 : 
12.5% (77 = 


3) 

; 1 ) 


43.4 % a (77 = 
37.0% a (77 = 


96) 

10 ) 


15.8% (77 : 
22.2% (77 ; 


: 35) 

6 ) 


30.8% (77 = 
25.9% (77 = 


68 ) 

7) 


10.0% ( 77 : 
14.8% (77 : 


: 22 ) 
= 4) 


44.8% a (77 = 78) 
42.1 % a (77 = 8) 


15.5% (77 = 27) 
21.1% (77 = 4) 


30.5% (77 = 53) 
15.8% (77 = 3) 


9.2% (77 = 16) 
21.1% (77 = 4) 


37.0% a (77 = 17) 
25.0% (77 = 2) 


17.4% (77 = 8 ) 
25.0% (77 = 2 ) 


32.6% (77 = 15) 
50.0% a (77 = 4) 


13.0% (77 = 6) 
0.0% (77 = 0) 
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Table 10 


Hispanic Respondents by Citizenship and Mono/Multiracial Self-Identification 


Citizenship status and ethnicity/race 

Format A 

Format B 

Format C 

None of the above 

All Hispanic respondents (ignoring 
citizenship) 

Monoracial Hispanic 

39.9% a (A=47) 

24.5% (A =30) 

27.9% (A =34) 

11.9% (A = 12) 

Multiracial Hispanic 

32.1% (A = 9) 

28.1% (A =6) 

47.8% a (A= 11) 

18.8% (A =3) 

U.S. citizenship only 

Monoracial Hispanic 

34.8% a (n = 24) 

24.6% (77 = 17) 

27.5% (77 = 19) 

13.0% (77 = 9) 

Multiracial Hispanic 

31.3% a (/1 = 5) 

18.8% (77 = 3) 

31.3% a (77 = 5) 

18.8% (77 = 3) 

U.S. + other citizenship 

Monoracial Hispanic 

27.8% (n = 5) 

27.8% (77 = 5) 

33.3% a (77 = 6) 

11.1% (77 =2) 

Multiracial Hispanic 

20.0% (/1 = 1) 

0.0% (77 = 0) 

80.0% a (77 = 4) 

0.0% (77 = 0) 

Other citizenship only 

Monoracial Hispanic 

50.0% a (77 = 18) 

22.2% (77 = 8) 

25.0% (77 = 9) 

2.8% (77 = 1) 

Multiracial Hispanic 

37.5% a (77 = 3) 

37.5% a (77 = 3) 

25.0% (77 = 2) 

0.0% (77 = 0) 


“Preferred format is in boldface. 


Table 11 


White Respondents by Citizenship and Mono/Multiracial Self-Identification 


Citizenship status and ethnicity/race 


Format A 


Format B 


Format C 


None of the above 


All White respondents (ignoring 
citizenship) 


Monoracial White 

45.2% 

a (A 

= 72) 

21.4% 

(A= 

= 34) 

25.4% (A = 

40) 

9.1% 

(N = 

= 14) 

Multiracial White 

23.1% 

(A = 

26) 

16.3% 

(N- 

= 12) 

64.1 % a (A 

= 74) 

10.0% 

(A = 

= 12) 

U.S. citizenship only 












Monoracial White 

41.9% 

(77 

= 26) 

19.4% 

(77 = 

= 12) 

29.0% (77 = 

18) 

9.7% 

(77 = 

6) 

Multiracial White 

18.3% 

(77 = 

17) 

6.5% 

(77 = 

= 6) 

66.7% a (77 = 

= 62) 

8.6% 

(77 = 

8) 

U.S. + other citizenship 












Monoracial White 

42.3% 

(77 

= 11) 

23.1% 

(77 = 

= 6) 

23.1% (77 = 

6) 

11.5% 

(77 = 

3) 

Multiracial White 

12.5% 

(77 = 

1) 

0.0% 

(77 = 

= 0) 

75.0% a (77 = 

= 6) 

12.5% 

(77 = 

1) 

Other citizenship only 












Monoracial White 

48.6% 

“(77 = 

= 35) 

22.2% 

(77 = 

= 16) 

22.2% (77 = 

16) 

6.9% 

(77 = 

5) 

Multiracial White 

34.8% 

(77 

= 8) 

26.1% 

(77 = 

= 6) 

26.1% (77 = 

6) 

13.0% 

(77 = 

3) 


“Preferred format is in boldface. 


How do the findings for format preferences explain the virtually universal preference of 
some minorities (namely Asian respondents and American Indians/Alaska Native respondents) 
for Fonnat C, regardless of mono/multiraciality? With respect to one’s question preference, just 
as mono/multiraciality supersedes the effects of U.S. acculturation, being able to identify one’s 
race/ethnicity by subcategory (and not just at a broad category level) supersedes the effects of 
mono/multiraciality. An example is being able to select Chinese rather than just Asian. Only 
Fonnat C asks or even allows American Indian/Alaska Native respondents to write in their tribal 
affiliations or Asian respondents (including Asian Americans respondents) to describe their 
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national origin (checkbox and write-in). One can think of tribal affiliation as a type of national 
origin, too, because many American Indian people think of tribes as nations. No such options to 
report ancestry/national origin are offered to White or Black respondents. When one compares 
Tables 13 and 15 (Asian respondents and American Indian/Alaska Native respondents, 
respectively) to Tables 11 and 12 (White and Black respondents, respectively), one sees that only 
multiracial White or multiracial Black U.S. citizens as a group prefer Fonnat C, whereas all Asian 
respondents and all American Indian/Alaska Native respondents as a group prefer Fonnat C. 

Table 12 

Black Respondents by Citizenship and Mono/Multiracial Self-Identification 


Citizenship status and ethnicity/race_Format A_Format B_Format C_None of the above 


All Black respondents (ignoring citizenship) 


Monoracial Black 

47.7% a (A = 80) 

20.8% (A =34) 

24.0% (A =40) 

11.8% (A= 14) 

Multiracial Black 

28.3% (A = 12) 

15.5% (A =4) 

53.9% a (A= 23) 

16.4% (A =5) 

U.S. citizenship only 

Monoracial Black 

47.2 % a {n = 50) 

22.6% (« = 24) 

23.6% (77 = 25) 

6.6% (77 = 7) 

Multiracial Black 

24.2% (n = 8) 

6.1% (n = 2) 

57.6% a (77 = 19) 

12.1% (77 =4) 

U.S. + other citizenship 

Monoracial Black 

52.4% a (« = 11) 

19.0% (/? =4) 

28.6% (77 = 6) 

0.0% (77 = 0) 

Multiracial Black 

33.3% a (n = 1) 

0.0% (77 = 0) 

33.3% a (77 = 1) 

33.3% (77 = 1) 

Other citizenship only 

Monoracial Black 

46.3 % a (« = 19) 

14.6% (77 = 6) 

22.0% (77 = 9) 

17.1% (77 =7) 

Multiracial Black 

37.5% a (n = 3) 

25.0% (77 =2) 

37.5% a (77 = 3) 

0.0% (77 = 0) 


“Preferred format is in boldface. 


Table 13 

Asian Respondents by Citizenship and Mono/Multiracial Self-Identification 


Citizenship status and ethnicity Format A Format B Format C None of the above 


All Asian respondents (ignoring citizenship) 


Monoracial Asian 

38.3% (A= 51) 

12.9% (A = 14) 

47.1 % a (A= 63) 

5.7% (A =6) 

Multiracial Asian 

23.8% (A =5) 

20.4% (A =4) 

53.6% a (A= 11) 

19.2% (A =3) 

U.S. citizenship only 

Monoracial Asian 

40.0% (77 = 18) 

8.9% (77 =4) 

48.9% a (77 = 22) 

2.2% (77 = 1) 

Multiracial Asian 

23.1% (77 = 3) 

23.1% (77 = 3) 

46.2% a (77 = 6) 

7.7% (77 = 1) 

U.S. + other citizenship 

Monoracial Asian 

27.3% (77 = 3) 

27.3% (77 = 3) 

45.5% a (77 = 5) 

0.0% (77 = 0) 

Multiracial Asian 

0.0% (77 = 0) 

0.0% (77 = 0) 

100.0% a (77 = 2) 

0.0% (77 = 0) 

Other citizenship only 

Monoracial Asian 

38.5% (77 = 30) 

9.0% (77 = 7) 

46.2% a (77 = 36) 

6.4% (77 = 5) 

Multiracial Asian 

25.0% (77 = 2) 

12.5% (77 = 1) 

37.5% a (77 = 3) 

25.0% (77 = 2) 


“Preferred format is in boldface. 
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Table 14 

Hawaiian/Pacific Islander Respondents by Citizenship and Mono/Multiracial Self- 
Identification 

None of the 

Citizenship status and ethnicity Format A Format B Format C above 

All Native Hawaiian/Pacific Islander respondents 
Monoracial Native Hawaiian/Pacific 
Islander 

Multiracial Native Hawaiian/Pacific 
Islander 

U.S. citizenship only 

Monoracial Native Hawaiian/Pacific 
Islander 

Multiracial Native Hawaiian/Pacific 
Islander 

Other citizenship only 

Multiracial Native Hawaiian/Pacific 
Islander 

“Preferred format is in boldface. 


(ignoring citizenship) 


0.0% 

(N= 

= 0) 

33.3% (N= 

= 1) 

67.7% a 

(N= 

= 2 ) 

0.0% (N = 

0) 

33.3% 

(N- 

= 1) 

0.0 %(N= 

= 0) 

33.3% 

(N = 

1) 

66.7% a (N 

= 2) 

0.0% 

(n = 

0) 

33.3% (/1 = 

= 1) 

67.7% a 

(77 = 

2) 

0.0% (77 = 

0) 

0.0% 

(n = 

0) 

0.0% (77 = 

0) 

0.0% 

(77 = 

0) 

100.0% a (77 

= 1) 

33.3% 

•(« 

= 1) 

1 0.0% (77 = 

0) 

33.3% a 

(77 = 

1) 

33.3% a (77 = 

= 1) 


Table 15 

American Indian/Alaska Native Respondents by Citizenship and Mono/Multiracial Self- 
Identification 


Citizenship status and ethnicity/race „ , r . „ _ , 

Format A Format B Format C above 


All American Indian/Alaska Native Respondents (ignoring citizenship) 


Monoracial Amer Ind / Alaska Nat 

30.6% 

(N= 

= 11) 

13.9% 

(N- 

= 5) 

55.0% a 

(N = 

= 20) 

5.6% 

(N= 

= 2) 

Multiracial Amer Ind / Alaska Nat 

21.5% 

(N= 

= 17) 

2.5% 

(N= 

= 2) 

72.9% a 

(N- 

= 61) 

4.9% 

(N= 

= 4) 

U.S. citizenship only 













Monoracial Amer Ind / Alaska Nat 

30.6% 

(77 = 

11) 

13.9% 

(77 = 

= 5) 

50.0% a 

(77 = 

= 18) 

5.6% 

(77 = 

2) 

Multiracial Amer Ind / Alaska Nat 

19.8% 

(77 = 

16) 

2.5% 

(77 = 

2) 

72.8% a 

(77 = 

= 59) 

4.9% 

(t? = 

4) 

U.S. + other citizenship 













Multiracial Amer Ind / Alaska Nat 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

100 .0% 

(77 

= 2) 

0.0% 

(77 = 

0) 

Monoracial Amer Ind / Alaska Nat 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

100 .0% 

(77 

= 1) 

0.0% 

(77 = 

0) 

Other citizenship only 













Monoracial Amer Ind / Alaska Nat 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

0.0% ( 

[n = 

0) 

0.0% 

(77 = 

0) 

Multiracial Amer Ind / Alaska Nat 

50.0% 

3 (77 

= 1) 

0.0% 

(77 = 

0) 

50.0% a 

(n = 

= 1) 

0.0% 

(77 = 

0) 


Note. Amer Ind = American Indian; Alaska Nat = Alaska Native. 
“Preferred format is in boldface. 


Hispanic U.S. citizens and Hawaiian/Pacific Islander respondents represent a possible 
exception to the effect that the provision of subcategorization options has on the format 
preferences of respondents regardless of how U.S.-acculturated they are. The sample sizes 
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for Hawaiian/Pacific Islander respondents are too small to draw any conclusions (see Table 
14). Fonnats B and C, but not A, pennit a Hispanic respondent to write in national origin 
(compare Figures 14 and 15 with Figure 13). Yet Hispanic U.S.-only citizens tend to prefer 
Fonnat A (see Table 9). However, a closer look at the Hispanic American group tells a 
somewhat more complex story. Although sample sizes are somewhat small, multiracial 
Hispanic respondents who have U.S. citizenship (sole or dual) do prefer Format C at least as 
much as Format A (see Table 10), which arguably could support the notion of a 
mono/multiraciality effect on fonnat preferences. Nevertheless, for Hispanic respondents, we 
do not observe the trend for preferring Fonnat C that we observe for Asian respondents or 
American Indian/Alaska Native respondents (compare Table 10 to Tables 13 and 15). Based 
on findings discussed in further detail below, we have concluded that many Hispanic U.S. 
citizens (including those who non-Hispanic respondents might consider to be multiracial) 
primarily self-identify with a Hispanic identity that cuts across U.S. notions of race, and 
asking them to consider race separately from their Hispanic identity causes them confusion 
and/or discomfort. An unreceptive or ambivalent attitude toward race as an additional 
identity may contribute to Hispanic respondents’ lukewann responses to anything other than 
a single question that treats Hispanic origin the same as race. For open-ended questions that 
had exceedingly low response rates in general as well as within specific racial and ethnic 
groups (see further discussion in the section Issues Raised With Regard to the U.S.-Style 
Questions), Hispanic respondents’ most widely expressed opinions concerned 
misunderstanding and discomfort over what to select for race after self-identifying as being 
of Hispanic origin. Moreover, 3.6% of Hispanic participants (and 3.2% of Hispanic 
participants who were U.S.-only citizens) expressly indicated that they wanted the two 
questions about race and ethnicity in Format B to be collapsed into a single question. For 
other open-ended questions that had very low overall response rates, 2.4% of Hispanic 
participants (and 3.2% of Hispanic participants who were U.S.-only citizens) expressly 
indicated that they wanted the two questions about race and ethnicity in Format C to be 
collapsed into a single question. In addition, while 34% of Hispanic U.S.-only citizens 
indicated a desire to change Format A, 42% indicated a desire to change Fonnat B, and 44% 
indicated a desire to change Format C. Out of the 26 sets of comments from Hispanic U.S.- 
only citizens regarding their understanding of Format C, 17 (65%) explicitly indicated 
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confusion about how to identify race separately from Hispanic origin. As one Hispanic U.S. 
citizen in this study remarked, “As a Hispanic, it seems wrong to separate us into White, 

Black, Asian, and so on. When asked, I say that I’m Latin, not White Latin, or Black Latin, 
or whatever.” Hispanic U.S.-only citizens’ comments regarding comfort levels with Fonnat 
C as well as their comments regarding Format B reinforce the notion that many Hispanic 
respondents prefer a single question that treats Hispanic origin similar to race (i.e., Format 
A). This conclusion is consistent with previous research that has shown that Hispanic 
respondents generally do not identify with racial groups (see discussion in Revisions to the 
Standards for the Classification of Federal Data on Race and Ethnicity, 1997). The U.S. 

Census Bureau’s attempts to encourage this identification by adding additional explanatory 
language to questions about race and ethnicity have been unsuccessful; the U.S. Census 
Bureau concluded, “[EJfforts to instruct respondents about the distinction between race and 
Hispanic origin may not be worthwhile” (Martin, 2006, p. 26). 

In summary, the evidence indicates that U.S. acculturation (being a U.S. citizen, being 
bom in the United States, and/or being a permanent U.S. resident) is related to a preference for a 
more pluralistic fonnat for the question about race and ethnicity exemplified by Fonnat C. 
Registrants who are not U.S.-acculturated generally preferred a simpler design like Fonnat A. 
Regardless of one’s U.S. acculturation (or lack thereof), being monoracial is related to being 
inclined toward a simpler Format exemplified by Format A, and being multiracial is related to 
being more inclined toward a more pluralistic format. Although Hispanic registrants often did 
not want racial and ethnic choices in addition to those for Hispanic origin, other registrants who 
could, conceivably, self-identify their race and ethnicity more specifically (such as in the form of 
national origin or tribal affiliation) generally preferred a more pluralistic question. 

Issues raised with regard to the U.S.-style questions. After each of the three formats 
for the U.S.-style questions about race and ethnicity (Fonnats A, B, and C), the final survey 
asked respondents to indicate in an open-ended response if and why they did not understand the 
question fonnat, were made uncomfortable by the question fonnat, or wanted to eliminate or 
change the question format. The lower halves of Figures 13-15 depict the specific questions. 
After data collection ended, research assistants coded the comments of the 720 respondents on 
the following dimensions: respondents’ understanding of the formats, comfort with the fonnats, 
and desired changes to the formats. In order to make a potentially onerous coding task more 
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manageable, the coders were allowed to select up to three qualitative codes to describe each 
response (see Tables 16-24). Although there were hardly any instances in which coders selected 
three codes (less than 1.3% of the time for all dimensions for all formats), it is possible that for 
some responses coders would have selected more than three codes if given the opportunity to do 
so. Therefore, some of the frequencies reported in Tables 16-24 might underreport participants’ 
concerns. In order to determine coder accuracy, the four different coders each independently 
recoded responses for 30 of the 720 participants, resulting in recoding for 120 participants in 
total (17%). Coder agreement between the first round of coding and the recoding was extremely 
high. The lowest agreement for any code was 95%, and coders achieved perfect agreement for 
the substantial majority of codes. When coders disagreed, it was usually over the specific nature 
of any changes that respondents described that they wanted to be made to a question format. 
Even here there was perfect agreement between coders approximately half of the time. 

Tables 16-24 set forth each code used (i.e., each issue) as well as the percentages and 
frequencies of respondents who made a comment that each code paraphrases (overall and by 
citizenship status). We omitted from the tables any codes that we had anticipated would 
summarize at least some of the participants’ comments but that we later discovered did not (i.e., 
whenever the percentage would equal zero in the column % who raised issue [N = 720]). The 
most frequently raised issues (generally the two most often mentioned for each tabled group or 
column) are highlighted in boldface. A relatively small percentage of the participants provided 
any responses to the open-ended questions. With regard to failure to understand questions, only 
10% of all participants indicated whether or not they understood Format A, only 12% Fonnat B, 
and only 13% Fonnat C (see Tables 16, 19, and 22, respectively). Response rates regarding 
discomfort with question formats were somewhat better—24% for Format A, 23% for Fonnat B, 
and 27% for Format C (see Tables 17, 20, and 23, respectively). Response rates regarding the 
necessity of a change to the questions were 38% for Format A, 38% for Fonnat B, and 32% for 
Format C (see Tables 18, 21, and 24, respectively). We do not know whether or the extent to 
which the nonresponses were due to general understanding and comfort with the formats, apathy 
toward the questions, other factors, or some combination of these reasons. Therefore, the 
practical significance of specific substantive comments from those who did provide a response is 
unclear. 
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Table 16 


Issues Raised With Regard to Understanding Format A 


Issue (summarized comments) 

% who raised issue 
(N = 720) 

% of those who raised the issue, by citizenship status 

U.S. U.S. +other Other 

(A = 388) (A =72) (A = 260) 

Yes, I understand the question 

4.7% (77 = 34) 

3.4% ( 77 = 13) 

4.2% (77 = 3) 

6.9% (77 = 18) 

No, 1 do not understand the question 

5.4% (;? = 39) 

4.1% (77 =16) 

5.6% (77 =4) 

7.3% (77 = 19) 

No response 

89.9% (77 = 647) 

92.5% (77 = 359) 

90.3% (77 = 65) 

85.8% (77 = 223) 

Don’t understand what African American means 

0.1% (77 = 1) 

0.0% (77 = 0) 

0.0% (77 = 0) 

0.4% (77 = 1) 

Hard to understand if not American citizen/Question 

0.8% (77 = 6) 

0.0% (77 = 0) 

0.0% (77 = 0) 

2.3% a (77 = 6) 

constructed for American citizens 

Asian doesn’t represent all Asians 

0.4% (77 = 3) 

0.0% (77 = 0) 

1.4% (77 = 1) 

0.8% (77 =2) 

What if you’re African? 

0.1% (77 = 1) 

0.0% (77 = 0) 

0.0% (77 = 0) 

0.4% (77 = 1) 

How do you describe yourself? might be different from your 

0.3% (77 =2) 

0.5% (77 = 2) 

0.0% (77 = 0) 

0.0% (77 = 0) 

race 

Don’t understand difference between Hispanic, Latino, and 

0.1% (77 = 1) 

0.3% (77 = 1) 

0.0% (77 = 0) 

0.0% (77 = 0) 

Spanish 

What to select if multiple races? 

1.4% a (77 =10) 

1.8% a (77 = 7) 

1.4% (77 = 1) 

0.8% (77 =2) 

Race/subchoice isn’t included in choices (Example: What to 

1.3% a (77 = 9) 

0.8% a (77 = 3) 

4.2% a (77 = 3) 

1.2% a (77 = 3) 

put if Middle Eastern?) 

Does race mean national or ethnic identity? 

0.3% (77 =2) 

0.0% (77 = 0) 

1.4% (77 = 1) 

0.4% (77 = 1) 

The answers mix racial categories with nationality 

0.3% (77 =2) 

0.0% (77 = 0) 

1.4% (77 = 1) 

0.4% (77 = 1) 

categories 

If chose response with “/,”does it mean you’re both items on 

0.1% (77 = 1) 

0.0% (77 = 0) 

0.0% (77 = 0) 

0.4% (77 = 1) 

either side of “/”? 

How do you describe yourself? is too vague 

0.4% (77 = 3) 

0.5% (77 = 2) 

0.0% (77 = 0) 

0.4% (77 = 1) 


Note. N = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column); n = number of 
participants with that citizenship status who expressed the comment in the first column. If fewer than two issues have an n that is 
greater than 1, then fewer than two percentages are in boldface. Each percentage is based on the total sample size (N) for the column 
in which it appears. 

“Percentages for the two most frequently raised issues are in boldface. 



Table 17 

Issues Raised With Regard to Comfort With Format A 


LtJ 
SO 



% who raised 

% of those who raised the issue, by citizenship status 

Issue (summarized comments) 

issue 

(N = 720) 

U.S. (TV = 388) 

U.S. + other (N = 72) 

Other (N = 260) 

Yes, I am uncomfortable with the question 

18.5% (n= 133) 

18.0% (n = 70) 

19.4% (h= 14) 

18.8% (n = 49) 

No, I am not uncomfortable with the question/1 am comfortable with the 
question 

5.1% (n = 37) 

4.4% (n = 17) 

2.8% (n = 2) 

6.9% (77 = 18) 

No response 

76.4% (n = 550) 

77.6% (n = 301) 

77.8% (n = 56) 

74.2% («= 193) 

Question constructed for American citizens 

1.0% (n = 7) 

0.0% (n = 0) 

1.4% ( n = 1) 

2.3% (« = 6) 

Uncomfortable not being able to clarify answer of Other 

0.3% (« = 2) 

0.3% ( n = 1) 

0.0% (n = 0) 

0.4% ( n = 1) 

Skin color is not the determiner of race 

0.3% (« = 2) 

0.0% ( n = 0) 

0.0% (n = 0) 

0.8% (« = 2) 

White and Black aren’t ancestries 

0.6% (n = 4) 

0.8% ( n = 3) 

0.0% (n = 0) 

0.4% (n = 1) 

How do yon describe yourself? doesn’t just mean 

What is the color of your skin?" 

0.7% ( n = 5) 

1% (n = 4) 

0.0% (n = 0) 

0.4% (n = 1) 

Not comfortable with Hispanic categories being interspersed in list 

0.7% (« = 5) 

0.3% (n = 1) 

2.8% (n = 2) 

0.8% (« = 2) 

What to select if multiple races? 

6.1% a (/? = 44) 

9.3% a ( n = 36) 

6.9% a ( n = 5) 

1.2% (n = 3) 

American Indian should be Native American 

0 .1% (n = \) 

0.3% ( n = 1) 

0.0% («=0) 

0.0% (n = 0) 

Uncomfortable with the word Hispanic 

0.3% (n = 2) 

0.0% ( n = 0) 

0.0% («=0) 

0.8% (n = 2) 

Should have the option to not select any race 

0.4% (n = 3) 

0.5% ( n = 2) 

0.0% («=0) 

0.4% (n = 1) 

Uncomfortable having to select Other 

1.1% (n = 8) 

0.8% ( n = 3) 

2.8% (n = 2) 

1.2% (n = 3) 

Not comfortable with Puerto Ricans, etc., having own category 

0.8% (n = 6) 

0.8% ( n = 3) 

0.0% («=0) 

1.2% (n = 3) 

Uncomfortable being asked about race or ethnicity 

1.8% ( n = 13) 

1.3% (n = 5) 

0.0% (77 = 0) 

3.1 % a (/? = 8) 

Race/subchoice isn’t included in choices 

(Example: What to put if Middle Eastern?) 

2.9% a (n =21) 

2.1 % a (n = 8) 

4.2 % a (n = 3) 

3.8% a (n= 10) 

Redundancy in answer choices 

0 . 1 % (« = 1) 

0.0% (n = 0) 

0.0% («=0) 

0.4% (« = 1) 

Uncomfortable with separation of Hispanic, Latino, or Spanish origin 

0 .1% {n = 1 ) 

0.0% ( n = 0) 

1.4% (n = 1) 

0.0% (n = 0) 

If chose response with “/”, does it mean you’re both items on 
either side of “/”? 

0.6% (n = 4) 

0.3% (n = 1) 

2.8% (n = 2) 

0.4% (n = 1) 

Stereotype threat 

0.7% (« = 5) 

0.5% (« = 2) 

0.0% («=0) 

1.2% (n = 3) 

Uncomfortable with use of the word Black or White 

1.1% (n = 8) 

0.8% (« = 3) 

1.4% (n = 1) 

1.5% (n = 4) 


AfoTe. ;V = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column); n = number of 
participants with that citizenship status who expressed the comment in the first column. Each percentage is based on the total sample 
size (AO for the column in which it appears. 

“Percentages for the two most frequently raised issues are in boldface. 



Table 18 


Issues Raised With Regard to Desire to Change (or Eliminate) Format A 


Issue (summarized comments) 

% who raised issue 
(N = 720) 

% of those who raised the issue, by citizenship status 

U.S. U.S.+other Other 

(N = 388) (N = 72) (N = 260) 

Yes, I would like the question to be changed 

33.8% 

(n = 

243) 

34.8% 

(» = 

135) 

26.4% 

(77 ; 

= 19) 

34.2% 

(77 

= 89) 

No, I don’t think the question needs to be changed 

3.9% 

(» = 

28) 

3.6% 

(n = 

14) 

4.2% 

(77 = 

= 3) 

4.2% 

(77 = 

= 11) 

No response 

62.4% 

(n = 

449) 

61.6% 

(n = 

239) 

69.4% 

(77 ^ 

= 50) 

61.5% 

(77 

= 160) 

Have options for people who are not American 

2.4% 

(n = 

17) 

1.5% 

(n = 

6) 

1.4% 

(77 = 

= 1) 

3.8% 

(77 = 

= 10) 

Eliminate the word Chicano (it’s offensive) 

0.4% 

(n = 

3) 

0.5% 

(n = 

2) 

0.0% 

(77 = 

= 0) 

0.4% 

(77 = 

= 1) 

Let Other specify what they are 

2.5% 

(» = 

18) 

2.1% 

(n = 

8) 

5.6% 

(77 = 

= 4) 

2.3% 

(77 = 

= 6) 

Change the phrase American Indian to clearly eliminate people 
from the Indian subcontinent (or clarify what you mean) 

0.1% 

(» = 

1) 

0.3% 

(n = 

1) 

0.0% 

(77 = 

= 0) 

0.0% 

(77 = 

= 0) 

Include fewer choices (it’s too intrusive) 

0.3% 

(« = 

2) 

0.3% 

0 n = 

1) 

0.0% 

(77 = 

= 0) 

0.4% 

(77 = 

= 1) 

Let people just list all of their races 

1.7% 

(« = 

12) 

2.1% 

(n = 

8) 

1.4% 

(77 = 

= 1) 

1.2% 

(77 = 

= 3) 

Add a Mestizo option 

0.1% 

(« = 

1) 

0.3% 

( n = 

1) 

0.0% 

(77 = 

= 0) 

0.0% 

(77 = 

= 0) 

Allow for multiple races 

11.8% 

“(» = 

= 85) 

18.6% 

d {n-- 

= 72) 

8.3% : 

An 

= 6) 

2.7% 

(77 = 

= 7) 

Use Native American Indian rather than American Indian 

0.4% 

(n = 

3) 

0.8% 

(n = 

3) 

0.0% 

(77 = 

= 0) 

0.0% 

(77 = 

= 0) 

Allow people to describe themselves according to nationality 

2.4% 

(« = 

17) 

0.8% 

(n = 

3) 

0.0% 

(77 = 

= 0 ) 

5.4% : 

An 

= 14) 

Allow people to opt out 

0.4% 

(« = 

3) 

0.3% 

(n = 

1) 

0.0% 

(77 = 

= 0 ) 

0.8% 

(77 = 

= 2 ) 

Don’t make Puerto Rican and Mexican their own categories 

2.4% 

(« = 

17) 

1.8% 

(n = 

7) 

4.2% 

(77 = 

= 3) 

2.7% 

(77 = 

= 7) 

Include a description of the purpose of the question 

0.4% 

(« = 

3) 

0.5% 

(n = 

2) 

0.0% 

(77 = 

= 0 ) 

0.4% 

(77 = 

= 1 ) 

Include more races or subchoices 

6.5% : 

*(« = 

= 47) 

5.2%‘ 

An = 

= 20 ) 

6.9% 

An 

= 5) 

8.5% : 

An 

= 22 ) 

Remove question altogether 

1.8% 

(« = 

13) 

1.8% 

(n = 

7) 

0.0% 

(77 = 

= 0 ) 

2.3% 

(77 = 

= 6 ) 

Rephrase How do you describe yourself? 

1.8% 

(« = 

13) 

2.3% 

(n = 

9) 

1.4% 

(77 = 

= 1) 

1.2% 

(77 = 

= 3) 

Separate Asian and Asian American (or Black or African American) 

1.8% 

(« = 

13) 

0.5% 

C n = 

2) 

1.4% 

(77 = 

= 1) 

3.8% 

(77 = 

= 10) 

Ask questions about race after the test, not before 

0.1% 

(n = 

1) 

0.3% 

(n = 

1) 

0.0% 

(77 = 

= 0) 

0.0% 

(77 = 

= 0) 

Change the formatting of the text boxes 

0.3% 

(n = 

2) 

0.3% 

(n = 

1) 

0.0% 

(77 = 

= 0) 

0.4% 

(77 = 

= 1) 

Eliminate the word Black or White 

1.3% 

(n = 

9) 

0.8% 

(n = 

3) 

1.4% 

(77 = 

= 1) 

1.9% 

(77 = 

= 5) 


Note. N = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column); n = number of 
participants with that citizenship status who expressed the comment in the first column. Each percentage is based on the total sample 
size (TV) for the column in which it appears. 

“Percentages for the two most frequently raised issues are in boldface. 



Table 19 

Issues Raised With Regard to Understanding Format B 




% of those who raised the issue, by citizenship status 

Issue (summarized comments) 

% who raised issue 
(N = 720) 

U.S. 

(N = 388) 

U.S. + other 
(N = 72) 

Other 
(N = 260) 

Yes, 1 understand the question 

No, I do not understand the question 

No response 

4.2% (,n = 30) 
7.9% (n = 57) 
87.9% (n = 633) 

3.9% (n = 15) 
7.0% (n = 27) 
89.2% (n = 346) 

2.8% (/1 = 2) 
12.5% (/1 = 9) 
84.7% (/7 = 61) 

5.0% (77 = 13) 
8.1% (77 =21) 
86.9% (77 = 226) 


Don’t understand what African American means 

Hard to understand if not American citizen/Question constructed for American citizens 
What if you’re African? 

Unclear if should answer both questions 

How do you describe yourself! might be different from your race 

If answer to first question is Hispanic, don’t understand how to answer second question 

Don’t understand difference between Hispanic, Latino, and Spanish 

Why need to know Hispanic, Latino, or Spanish origin? 

Don’t understand why Hispanic origins are not races 
What to select if multiple races? 

Don’t understand purpose of question (why need this information?) 

Race/subchoice isn’t included in choices (Example: What to put if Middle Eastern?) 
Does race mean national or ethnic identity? 

The answers mix racial categories with nationality categories 
Redundancy in question 

If answer something with a slash, does it mean you’re both items on either side of the 
slash? (Example: If answer Black/African American, are you both? What if you’re a 
Black African?) 

How do you describe yourself? is too vague 


0.1% 

(77 = 

1) 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

0.4% 

(77 = 

1) 

0.6% 

(77 = 

4) 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

1.5% 

(77 

= 4) 

0.3% 

(77 = 

2 ) 

0.3% 

(77 = 

1) 

0.0% 

(77 = 

0) 

0.4% 

(77 = 

1) 

0.1% 

(77 = 

1) 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

0.4% 

(77 = 

1) 

0.6% 

(77 = 

4) 

0.5% 

(77 = 

2 ) 

0.0% 

(77 = 

0) 

0.8% 

(77 = 

2 ) 

1.4% 

(77 

= 10) 

1.3% 

(77 = 

5) 

4.2% 

(77 

= 3) 

0.8% 

(77 = 

2 ) 

0.4% 

(77 = 

3) 

0.0% 

(77 = 

0) 

1.4% 

(77 = 

1) 

0.8% 

(77 = 

2 ) 

0.6% 

(77 = 

4) 

0.5% 

(77 = 

2 ) 

0.0% 

(77 = 

0) 

0.8% 

(77 = 

2 ) 

0.4% 

(77 = 

3) 

0.3% 

(77 = 

1) 

1.4% 

(77 = 

1) 

0.4% 

(77 = 

1) 

1 .1% 

(77 

= 8) 

1.5% 

(77 

= 6) 

2 .8% 

(77 

= 2) 

0.0% 

(77 = 

0) 

0.1% 

(77 = 

1) 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

0.4% 

(77 = 

1) 

1.1% 

(77 = 

8) 

0.5% 

(77 = 

2) 

1.4% 

(77 = 

1) 

1.9% 

(77 

= 5) 

0.3% 

(77 = 

2 ) 

0.0% 

(77 = 

0) 

1.4% 

(77 = 

1) 

0.4% 

(77 = 

1) 

0.1% 

(77 = 

1) 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

0.4% 

(77 = 

1) 

0.3% 

(77 = 

2 ) 

0.3% 

(77 = 

1) 

0.0% 

(77 = 

0) 

0.4% 

(77 = 

1) 

0.1% 

(77 = 

1) 

0.0% 

(77 = 

0) 

0.0% 

(77 = 

0) 

0.4% 

(77 = 

1) 

1.0% 

(77 = 

7) 

1.5% 

(77 

= 6) 

0.0% 

(77 = 

0) 

0.4% 

(77 = 

1) 


Note. N = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column); n = number of 
participants with that citizenship status who expressed the comment in the first column. Each percentage is based on the total sample 
size (TV) for the column in which it appears. 

“Percentages for the two most frequently raised issues for each column are in boldface. 



Table 20 

Issues Raised With Regard to Comfort With Format B 



% who raised 

% of those who raised the issue, by citizenship status 


issue 

U.S. 

U.S. + other 

Other 

Issue (summarized comments) 

(N = 720) 

(N = 388) 

(N = 72) 

(N = 260) 

Yes, I am uncomfortable with the question 

No, I am not uncomfortable with the question/1 am comfortable with the question 

No response 

19.0% (n= 137) 
4.4% ( n = 32) 
76.5% (/? = 551) 

19.8% (77 = 77) 
3.6% (77 = 14) 
76.5% (77 = 297) 

16.7% (77= 12) 
1.4% (77 = 1) 
81.9% (77 = 59) 

18.5% (77 = 48) 
6.5% (77= 17) 
75.0% (77 = 195) 


Question constructed for American citizens 


1.1% 

(77 = 

8) 

0.0% 

(77 = 

0) 

1.4% (77 = 

1) 

2.7% (77 = 

7) 

Skin color is not the determiner of race 


0.3% 

(77 = 

2) 

0.3% 

(77 = 

1) 

0.0% (77 = 

0) 

0.4% (77 = 

1) 

White and Black aren’t ancestries 


0.1% 

(77 = 

1) 

0.0% 

(77 = 

0) 

0.0% (77 = 

0) 

0.4% (77 = 

1) 

Black and White are colors, not races 


0.1% 

(77 = 

1) 

0.3% 

(77 = 

1) 

0.0% (77 = 

0) 

0.0% (77 = 

0) 

How do you describe yourself? doesn’t just mean What is the color of your 

skin? 

0.4% 

(77 = 

3) 

0.3% 

(77 = 

1) 

0.0% (77 = 

0) 

0.8% (77 = 

2) 

Some Hispanics don’t identify with another race 


0.3% 

(77 = 

2) 

0.5% 

(77 = 

2) 

0.0% (77 = 

0) 

0.0% (77 = 

0) 

What to select if multiple races? 


4.3% 

(77 

= 31) 

7.2% 

(77 

= 28) 

1.4% (77 = 

1) 

0.8% (77 = 

2) 

American Indian should be Native American 


0.1% 

(77 = 

1) 

0.3% 

(77 = 

1) 

0.0% (77 = 

0) 

0.0% (77 = 

0) 

Uncomfortable with the word Hispanic 


0.3% 

(77 = 

2) 

0.3% 

(77 = 

1) 

0.0% (77 = 

0) 

0.4% (77 = 

1) 

Should have the option to not select any race 


0.3% 

(77 = 

2) 

0.3% 

(77 = 

1) 

0.0% (77 = 

0) 

0.4% (77 = 

1) 

Uncomfortable having to select Other 


0.7% 

(77 = 

5) 

0.5% 

(77 = 

2) 

2.8% a (77 

= 2) 

0.4% (77 = 

1) 

Not comfortable with Puerto Ricans, etc., having own category 


0.6% 

(77 = 

4) 

0.8% 

(77 = 

3) 

0.0% (77 = 

0) 

0.4% (77 = 

1) 

Uncomfortable being asked about race or ethnicity 


1.4% 

(77 = 

10) 

1.3% 

(77 = 

5) 

0.0% (77 = 

0) 

1.9% (77 = 

5) 

Race/subchoice isn’t included in choices (Example: What to put if Middle Eastern?) 

3.2% 

(77 = 

23) 

2.3% 

(77 = 

9) 

2.8% a (77 

= 2) 

4.6% a (77 

= 12) 

Redundancy in answer choices 


0.6% 

(77 = 

4) 

0.0% 

(77 = 

0) 

0.0% (77 = 

0) 

1.5% (77 = 

4) 

Uncomfortable with separation of Hispanic, Latino, or Spanish origin 


4.2% 

(77 

= 30) 

3.6% 

(77 

= 14) 

5.6% a (77 

= 4) 

4.6% a (77 

= 12) 

If chose response with “/,”does it mean you’re both items on either side of' 


0.3% 

(77 = 

2 ) 

0.3% 

(77 = 

1) 

0.0% (77 = 

0) 

0.4% (77 = 

1) 

Stereotype threat 


0.7% 

(77 = 

5) 

1.0% 

(77 = 

4) 

0.0% (77 = 

0) 

0.4% (77 = 

1) 

Uncomfortable with the word race 


0.1% 

(77 = 

1) 

0.3% 

(77 = 

1) 

0.0% (77 = 

0) 

0.0% (77 = 

0) 

Uncomfortable with use of the word Black or White 


1.1% 

(77 = 

8) 

0.8% 

(77 = 

3) 

1.4% (77 = 

1) 

1.5% (77 = 

4) 


Note. N = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column); n = number of 
participants with that citizenship status who expressed the comment in the first column. In each column, more than two percentages 
are in boldface in the case of ties. Each percentage is based on the total sample size (AO for the column in which it appears. 
“Percentages for the two most frequently raised issues for each column are in boldface. 



Table 21 

Issues Raised With Regard to Desire to Change (or Eliminate) Format B 


% of those who raised the issue, by citizenship status 
% who raised issue U.S. U.S. + other Other 


Issue (summarized comments) 

(N = 720) 

(N = 388) 

(N 

= 72) 

(N = 260) 

Yes, I would like the question to be changed 

35.1% (n = 

253) 

36.3% (77 = 

: 141) 

33.3% 

(77 = 

! 24) 

33.8% (77 = 

88) 

No, I don’t think the question needs to be changed 

2.9% (n = 

= 21) 

3.4% (77 = 

= 13) 

1.4% 

(77 = 

= 1) 

2.7% (77 = 

-1) 

No response 

61.9% (n = 

446) 

60.3% (77 = 

234) 

65.3% 

(77 = 

47) 

63.5% (77 = 

165) 

Put races in alphabetical order 

0.1% (77 = 

1) 

0.0% (77 = 

0) 

0.0% 

(77 = 

0) 

0.4% (77 = 

1) 

Have options for people who are not American 

1.1% (77 = 

8) 

0.3% (77 = 

1) 

1.4% 

(77 = 

1) 

2.3% (77 = 

6) 

Eliminate the word Chicano (it’s offensive) 

0.3% (77 = 

2) 

0.3% (77 = 

1) 

0.0% 

(77 = 

0) 

0.4% (77 = 

1) 

Let Other specify what they are 

0.7% (77 = 

5) 

0.8% (77 = 

3) 

1.4% 

(77 = 

1) 

0.4% (77 = 

1) 

Hispanic should be listed for Question 2 as well as Question 1 

0.7% (77 = 

5) 

1% (77 = 4) 

0.0% 

(77 = 

0) 

0.4% (77 = 

1) 

Hispanic origins should be considered races 

0.1% (77 = 

1) 

0.3% (77 = 

1) 

0.0% 

(77 = 

0) 

0.0% (77 = 

0) 

Change the phrase American Indian to clearly eliminate people from the Indian 

0.3% (77 = 

2) 

0.3% (77 = 

1) 

0.0% 

(77 = 

0) 

0.4% (77 = 

1) 

subcontinent (or clarify what you mean) 










Include fewer choices (it’s too intrusive) 

0.7% (77 = 

5) 

0.8% (77 = 

3) 

1.4% 

(77 = 

1) 

0.4% (77 = 

1) 

Let people just list all of their races 

1.4% (77 = 

10) 

1% (77 = 4) 

2.8% 

(77 = 

2) 

1.5% (77 = 

4) 

Add a Mestizo option 

0.1% (77 = 

1) 

0.3% (77 = 

1) 

0.0% 

(77 = 

0) 

0.0% (77 = 

0) 

Allow for multiple races 

7.8% a (77 

= 56) 

11.9%“ (77 

= 46) 

6.9% 

(77 

= 5) 

1.9% (77 = 

5) 

Use Native American Indian rather than American Indian 

0.4% (77 = 

3 ) 

0.8% (77 = 

3 ) 

0.0% 

(77 = 

0) 

0.0% (77 = 

0) 

Allow people to describe themselves according to nationality 

2.2% (77 = 

16) 

1.3% (77 = 

5) 

0.0% 

(77 = 

0) 

4.2% (77 = 

11) 

Allow people to opt out 

0.4% (77 = 

3 ) 

0.3% (77 = 

1) 

0.0% 

(77 = 

0) 

0.8% (77 = 

2) 

Don’t make Puerto Rican and Mexican their own categories 

0.8% (77 = 

6) 

0.5% (77 = 

2) 

1.4% 

(77 = 

1) 

1.2% (77 = 

3 ) 

Include a description of the purpose of the question 

0.3% (77 = 

2) 

0.5% (77 = 

2) 

0.0% 

(77 = 

0) 

0.0% (77 = 

0) 

Include more races or subchoices 

6.9% a (77 

= 50) 

5.2% (77 = 

20) 

11.1% 

'•(« 

= 8) 

8.5% a (77 

= 22) 

Remove question altogether 

3.1% (77 = 

22) 

3.1% (77 = 

12) 

0.0% 

(77 = 

0) 

3.8% (77 = 

10) 

Rephrase How do you describe yourself? 

1.9% (77 = 

14) 

2.6% (77 = 

10) 

0.0% 

(77 = 

0) 

1.5% (77 = 

4) 

Separate Asian and Asian American (or Black or African American) 

1.3% (77 = 

9) 

0.5% (77 = 

2) 

0.0% 

(77 = 

0) 

2.7% (77 = 

7) 

Make all the choices one question (don’t separate Hispanic, Latin, or Spanish origin) 

6.3% (77 = 

45) 

7.2 % a (77 : 

= 28) 

2.8% 

(77 = 

2) 

5.8% a (77 

= 15) 

Ask questions about race after the test, not before 

0.3% (77 = 

2) 

0.5% (77 = 

2) 

0.0% 

(77 = 

0) 

0.0% (77 = 

0) 

Change the formatting of the text boxes 

0.3% (77 = 

2) 

0.5% (77 = 

2) 

0.0% 

(77 = 

0) 

0.0% (77 = 

0) 

Eliminate the word Black or White 

1.3% (77 = 

9) 

1.0% (77 = 

4) 

1.4% 

(77 = 

1) 

1.5% (77 = 

4) 


Note. N = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column), n = number of 
participants with that citizenship status who expressed the comment in the first column. Each percentage is based on the total sample 
size (AO for the column in which it appears. 

“Percentages for the two most frequently raised issues for each column are in boldface. 



Table 22 


Issues Raised With Regard to Understanding Format C 




% of those who raised the issue, by citizenship status 


% who raised issue 

U.S. 

U.S. + other 

Other 

Issue (summarized comments) 

(N = 720) 

(N = 388) 

(N = 72) 

(N = 260) 

Yes, 1 understand the question 

3.9% (n = 28) 

3.1% (n = 12) 

1.4% (n = 1) 

5.8% (77= 15) 

No, I do not understand the question 

9.4% (n = 68) 

9.3% (« = 36) 

11.1% (n = 8) 

9.2% (77 = 24) 

No response 

86.7% (n = 624) 

87.6% (n = 340) 

87.5% (n = 63) 

85.0% (77 = 221) 

Don’t understand what African American means 

0.3% (n = 2) 

0.0% (n = 0) 

1.4% (n = 1) 

0.4% (77 = 1) 

What if you’re African but not Black? 

0.1% (n = 1) 

0.0% (« = 0) 

0.0% (n = 0) 

0.4% (77 = 1) 

Hard to understand if not American citizen/Question constructed for 

0.3% (n = 2) 

0.0% ( n = 0) 

1.4% (n = 1) 

0.4% (77 = 1) 

American citizens 





How far back does ancestry go? 

0.7% (n = 5) 

1.0% (n = 4) 

0.0% (n = 0) 

0.4% (77 = 1) 

What if you’re African? 

0.6% (« = 4) 

0.0% ( n = 0) 

0.0% (n = 0) 

1.5% a (77 =4) 

If answer to the first question is Hispanic, don’t understand what to answer 

1.4% a (n = 10) 

2.6% a (n = 10) 

0.0% (n = 0) 

0.0% (77 = 0) 

second question 





Don’t understand difference between Hispanic, Latino, and Spanish 

0.6% (n = 4) 

0.3% (« = 1) 

0.0% (n = 0) 

1.2% (77 = 3) 

Why need to know Hispanic, Latino, or Spanish origin? 

0.3% (n = 2) 

0.5% (« = 2) 

0.0% (n = 0) 

0.0% (77 = 0) 

Don’t understand why Hispanic origins are not races 

0.8% (n = 6) 

0.8% ( n = 3) 

1.4% (n = 1) 

0.8% (77 = 2) 

What to select if multiple races? 

0.6% (« = 4) 

0.3% ( n = 1) 

1.4% (« = 1) 

0.8% (77 = 2) 

Don’t understand purpose of question (why need this information?) 

0.6% (« = 4) 

0.5% (« = 2) 

0.0% (77 = 0) 

0.8% (77 = 2) 

Race/subchoice isn’t included in choices (Example: What to put if Middle 

2.1 % a (n = 15) 

0.8% (« = 3) 

4.2 % a (77 = 3) 

3.5% a (77 = 9) 

Eastern?) 





Does race mean national or ethnic identity? 

0.7% (n = 5) 

1.3% a (n = 5) 

0.0% (77 = 0) 

0.0% (77 = 0) 

The answers mix racial categories with nationality categories 

0.3% (« = 2) 

0.5% (« = 2) 

0.0% (77 = 0) 

0.0% (77 = 0) 

Redundancy in question 

0.3% (n = 2) 

0.3% ( n = 1) 

0.0% (77 = 0) 

0.4% (77 = 1) 


Note. N = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column); n = number of 
participants with that citizenship status who expressed the comment in the first column. If fewer than two issues have an n that is 
greater than 1, then fewer than two percentages are in boldface. Each percentage is based on the total sample size (N) for the column 
in which it appears. 

“Percentages for the two most frequently raised issues for each column are in boldface. 


Table 23 


Issues Raised With Regard to Comfort With Format C 




% of those who raised the issue, by citizenship status 

Issue (summarized comments) 

% who raised issue 
( N = 720) 

U.S. 

(N = 388) 

U.S. + other 
(N = 72) 

Other 
(N = 260) 

Yes, I am uncomfortable with the question 

No, I am not uncomfortable with the question/1 am comfortable with the question 
No response 

19.2% (n = 138) 
5.0% (n = 36) 
75.7% (« = 545) 

20.1% (n = 78) 
3.4% (n = 13) 
76.5% (n = 297) 

16.7% (n = 12) 
5.6% (n = 4) 
76.4% (n = 55) 

18.5% (/1 = 48) 
7.3% (77 = 19) 
74.2% (77 = 193) 


Uncomfortable with fact that White is first (should be alphabetical) 

0.3% 

(77 = 

2 ) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0.4% (77 = 

1 ) 

Question constructed for American citizens 

0 .8% 

(77 = 

6 ) 

0 .8% 

(77 = 

3) 

0 .0% 

(77 = 

0 ) 

1 .2% (77 = 

3) 

Uncomfortable not being able to clarify answer of Other 

0.3% 

(77 = 

2 ) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0.4% (77 = 

1 ) 

White and Black aren’t ancestries 

0 .6% 

(77 = 

4) 

0.5% 

(77 = 

2 ) 

0 .0% 

(77 = 

0 ) 

0 .8% (77 = 

2 ) 

Black and White are colors, not races 

0.4% 

(77 = 

3) 

0.5% 

(77 = 

2 ) 

0 .0% 

(77 = 

0 ) 

0.4% (77 = 

1 ) 

Some Hispanics don’t identify with another race 

0 .8% 

(77 = 

6 ) 

1.3% 

(77 = 

5) 

0 .0% 

(77 = 

0 ) 

0.4% (77 = 

1 ) 

What to select if multiple races? 

1.3% 

(77 = 

9) 

2 .1% 

(77 = 

8 ) 

0 .0% 

(77 = 

0 ) 

0.4% (77 = 

1 ) 

American Indian should be Native American 

0 .1% 

(77 = 

1 ) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0 .0% (77 = 

0 ) 

Uncomfortable with the word Hispanic 

0 .1% 

(77 = 

1 ) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0 .0% (77 = 

0 ) 

Should have the option to not select any race 

0 .6% 

(77 = 

4) 

0 .8% 

(77 = 

3) 

0 .0% 

(77 = 

0 ) 

0.4% (77 = 

1 ) 

Uncomfortable having to select Other 

0.7% 

(77 = 

5) 

0.5% 

(77 = 

2 ) 

1.4% 

(77 = 

1) 

0 .8% (77 = 

2 ) 

Not comfortable with Puerto Ricans, etc., having own category 

0.4% 

(77 = 

3) 

0.5% 

(77 = 

2 ) 

0 .0% 

(77 = 

0 ) 

0.4% (77 = 

1 ) 

Uncomfortable being asked about race or ethnicity 

1.9% 

(77 = 

14) 

1.5% 

(77 = 

6 ) 

0 .0% 

(77 = 

0 ) 

3.1% a (77^ 

= 8 ) 

Race/subchoice isn’t included in choices (Example: What to put if Middle Eastern?) 

3.3% 

(77 

= 24) 

3.6% 

> 

= 14) 

2.8% 

(77 

= 2 ) 

3.1% a (77^ 

= 8 ) 

Redundancy in answer choices 

0 .1% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0 .0% 

(77 = 

0 ) 

0.4% (77 = 

1 ) 

Uncomfortable with separation of Hispanic, Latino, or Spanish origin 

3.6% 

(77 

= 26) 

3.9% 

a (77 

= 15) 

2.8% 

(77 

= 2) 

3.5% 3 (77 : 

= 9) 

If chose response with “/,” does it mean you’re both items on either side of “/”? 

0 .6% 

(77 = 

4) 

0.5% 

(77 = 

2 ) 

1.4% 

(77 = 

1) 

0.4% (77 = 

1 ) 

Uncomfortable with the phrasing Some other race 

0 .8% 

(77 = 

6 ) 

0 .8% 

(77 = 

3) 

1.4% 

(77 = 

1) 

0 .8% (77 = 

2 ) 

Stereotype threat 

0 .6% 

(77 = 

4) 

0.5% 

(77 = 

2 ) 

0 .0% 

(77 = 

0 ) 

0 .8% (77 = 

2 ) 

Uncomfortable specifying tribe for American Indian or Alaskan Native 

1 .0% 

(77 = 

7) 

1 .8% 

(77 = 

7) 

0 .0% 

(77 = 

0 ) 

0 .0% (77 = 

0 ) 

Uncomfortable with the word race 

1 .0% 

(77 = 

7) 

1 .0% 

(77 = 

4) 

0 .0% 

(77 = 

0 ) 

1 .2% (77 = 

3) 

Uncomfortable with use of the word Black or White 

1.1% 

(77 = 

8) 

0 .8% 

(77 = 

3) 

1.4% 

(77 = 

1) 

1.5% (77 = 

4) 


Note. N = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column); n = number of 
participants with that citizenship status who expressed the comment in the first column. In each column, more than two percentages 
are in boldface in the case of ties. Each percentage is based on the total sample size (TV) for the column in which it appears. 
“Percentages for the two most frequently raised issues for each column are in boldface. 



Table 24 

Issues Raised With Regard to Desire to Change (or Eliminate) Format C 

% who raised % of those who raised the issue, by citizenship status 
issue U.S. U.S.+other Other 


Issue (summarized comments) 

(N 

= 720) 

(N 

= 388) 

(N 

= 72) 

(N 

= 260) 

Yes, I would like the question to be changed 

28.5% 

(77 = 

= 205) 

26.5% (77 = 

= 103) 

22 .2% 

(77 = 

= 16) 

33.1% (77 = 

= 86) 

No, I don’t think the question needs to be changed 

3.8% 

(77 = 

= 27) 

3.6% (77 = 

= 14) 

5.6% 

(77 = 

= 4) 

3.5% (77 = 

= 9) 

No response 

67.8% 

(77 = 

- 488) 

69.8% (77 = 

= 271) 

72.2% 

(77 = 

= 52) 

63.5% 

(77 = 

= 165) 

Put races in alphabetical order 

0 .6% 

(77 = 

4) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

1 .2% 

(77 = 

3) 

Have options for people who are not American 

0.4% 

(77 = 

3) 

0.5% 

(77 = 

2 ) 

0 .0% 

(77 = 

0 ) 

0.4% 

(77 = 

1 ) 

Eliminate the word Chicano (it’s offensive) 

0 .1% 

(77 = 

1 ) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0 .0% 

(77 = 

0 ) 

Hispanic should be listed for Question 2 as well as Question 1 

1 .0% 

(77 = 

7) 

1 .0% 

(77 = 

4) 

0 .0% 

(77 = 

0 ) 

1 .2% 

(77 = 

3) 

Hispanic origins should be considered races 

1 .1% 

(77 = 

8 ) 

1 .0% 

(77 = 

4) 

0 .0% 

(77 = 

0 ) 

1.5% 

(77 = 

4) 

Change the phrase American Indian to clearly eliminate people from the Indian 

0.4% 

(77 = 

3) 

0 .8% 

(77 = 

3) 

0 .0% 

(77 = 

0 ) 

0 .0% 

(77 = 

0 ) 

subcontinent (or clarify what you mean) 













Include fewer choices (it’s too intrusive) 

2 .1% 

(77 = 

15) 

2 .1% 

(77 = 

8 ) 

1.4% 

(77 = 

1 ) 

2.3% 

(77 = 

6 ) 

Let people just list all of their races 

1.5% 

(77 = 

11 ) 

1.5% 

(77 = 

6 ) 

1.4% 

(77 = 

1 ) 

1.5% 

(77 = 

4) 

Add a Mestizo option 

0.3% 

(77 = 

2 ) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0.4% 

(77 = 

1 ) 

Allow for multiple races 

1 .1% 

(77 = 

8 ) 

1.5% 

(77 = 

6 ) 

0 .0% 

(77 = 

0 ) 

0 .8% 

(77 = 

2 ) 

Use Native American Indian rather than American Indian 

0 .1% 

(77 = 

1 ) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0 .0% 

(77 = 

0 ) 

Allow people to describe themselves according to nationality 

2.5% 

(77 = 

18) 

1 .0% 

(77 = 

4) 

0 .0% 

(77 = 

0 ) 

5.4% 

(77 

= 14) 

Allow people to opt out 

0 .8% 

(77 = 

6 ) 

0 .8% 

(77 = 

3) 

0 .0% 

(77 = 

0 ) 

1 .2% 

(77 = 

3) 

Don’t make Puerto Rican and Mexican their own categories 

0.3% 

(77 = 

2 ) 

0.5% 

(77 = 

2 ) 

0 .0% 

(77 = 

0 ) 

0 .0% 

(77 = 

0 ) 

Include a description of the purpose of the question 

0.7% 

(77 = 

5) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

1.5% 

(77 = 

4) 

Include more races or subchoices 

6.1% 

(77 

= 44) 

4.6% 

(77 

= 18) 

5.6% 

(77 

= 4) 

8.5% 

(77 

= 22) 

Remove question altogether 

3.3% 

(77 = 

24) 

3.4% 

(77 = 

13) 

0 .0% 

(77 = 

0 ) 

4.2% 

(77 = 

11 ) 

Rephrase How do you describe yourself? 

0.4% 

(77 = 

3) 

0 .8% 

(77 = 

3) 

0 .0% 

(77 = 

0 ) 

0 .0% 

(77 = 

0 ) 

Separate Asian and Asian American (or Black or African American) 

1 .0% 

(77 = 

7) 

0.5% 

(77 = 

2) 

0 .0% 

(77 = 

0 ) 

1.9% 

(77 = 

5) 

Make all the choices one question (don’t separate Hispanic, Latin, or Spanish origin) 

4.0% 

(77 

= 29) 

4.1% 

(77 

= 16) 

4.2% 

(77 

-3) 

3.8% 

(77 = 

10 ) 

Remove some in Some other race 

0 .8% 

(77 = 

6 ) 

0 .8% 

(77 = 

3) 

1.4% 

(77 = 

1 ) 

0 .8% 

(77 = 

2) 

Ask questions about race after the test, not before 

0 .1% 

(77 = 

1 ) 

0.3% 

(77 = 

1 ) 

0 .0% 

(77 = 

0 ) 

0 .0% 

(77 = 

0 ) 

Change the formatting of the text boxes 

0.4% 

(77 = 

3) 

0.5% 

(77 = 

2) 

0 .0% 

(77 = 

0 ) 

0.4% 

(77 = 

1 ) 

Eliminate the word Black or White 

1 . 1 % 

(77 = 

8 ) 

0.5% 

(77 = 

2 ) 

1.4% 

(77 = 

1 ) 

1.9% 

(77 = 

5 ) 


Note. N = total sample size for all respondents or for a particular citizenship status (i.e., total sample size for a column); n = number of 
participants with that citizenship status who expressed the comment in the first column. Each percentage is based on the total sample 
size (TV) for the column in which it appears. 

“Percentages for the two most frequently raised issues for each column are in boldface. 


What follows is a summary of the major findings, where percentages are conservatively 
based on the entire sample of 720 participants or, in the case of a subsample based on citizenship 
status, the entire subsample. (If one assumed instead that nonresponses did not reliably indicate 
understanding of, comfort with, and lack of desire to make any changes to question fonnats, then 
given the relatively low overall response rates to the open-ended questions—between 10% and 
39% [a low response in Dillman’s (2000) standards]—the percentages below could be 2 Vi to 10 
times as large.) In terms of misunderstanding Format A (Figure 13; Table 16), the most common 
comments (1.4% overall; 1.8% for U.S.-only citizens) were concerns about what to select if one 
identifies with more than one race or if one’s race and/or ethnicity is not included in the options 
(e.g.. Middle Eastern). For instance, one respondent (a U.S.-only citizen) stated, “I don’t 
understand the responses because they are limited to one race, and I describe myself as more 
than one race (White and Native American), which is why I selected Other.” These percentages 
jumped to 6.1% (9.3% for U.S.-only citizens) in terms of the discomfort people felt when they 
wanted to select more than one race but could not and to 2.9% (2.1% for U.S.-only citizens) if 
one’s race and/or ethnicity was not included in the options (see Table 17). One participant (a 
U.S.-only citizen) commented: 

I am not sure what selection applies to Middle Eastern or Arabs, which is my ethnic 
descent. I feel I can be classified as Asian or possibly White (and in some cases for Saudi 
Arabians, Egyptians, or UAE may feel like they are classified as African!). I feel 
uncomfortable selecting Other, so often I will just select White. 

As to whether Format A should be changed, the percentages again jumped for these two 
issues in particular: 11.8% (18.6% for U.S.-only citizens) of the respondents wanted changes 
made to allow multiracial responding, and 6.5% (5.2% for U.S.-only citizens) desired additional 
racial and ethnic groups (see Table 18). Only 1.8% of participants (also 1.8% of U.S.-only 
citizens) expressly indicated that they wanted Format A eliminated altogether (see Table 18). 

In terms of misunderstanding, discomfort with, and desire to change Fonnat B 
(Figure 14; Tables 19-21), respondents were concerned only to a slightly lesser extent with what 
to select if one identifies with more than one race or if one’s race and/or ethnicity is not included 
in the options. As with Fonnat A, these were the most commonly raised issues for Fonnat B. 
Furthermore, several persons (1.4% overall; 1.3% of U.S.-only citizens) stated that they did not 
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understand how to identify race in the second question of Format B if they identified as Hispanic 
in the first question of Format B (see Table 19). One dual U.S. citizen wrote: 

I always find this type of framework confusing. So it splits me up based on whether I am 
Hispanic or not, then asks me about my race? I guess this is a good way to get at Black 
Hispanics, for instance, or etc. But sometimes I wonder whether asking the top portion 
(about whether I am Hispanic or not) prevents me from answering the second section. 

In addition, 3.1% of respondents (and 3.1% of U.S.-only citizens) wanted Fonnat B 
eliminated altogether (see Table 21). 

In comparison to Fonnats A and B, there was notably less misunderstanding (0.6% 
overall; 0.3% for U.S.-only citizens), less discomfort (1.3% overall; 2.1% for U.S.-only citizens), 
and less desire to change (1.1% overall; 1.5% for U.S.-only citizens) Fonnat C (Figure 15) in 
tenns of the ability to report more than one race (see Tables 22-24). Given the pluralism of 
Format C, this is not surprising. As with Formats A and B, participants expressed concerns about 
what to report if a race and/or ethnicity was not explicitly provided as a response option, 
although the level of concern was lower for Format C (compare Tables 16 and 19 to Table 22). 
As with Format B, some participants indicated (a) that they did not understand how to identify 
race in the second question if they identified as Hispanic in the first question (1.4% overall; 2.6% 
for U.S.-only citizens; see Table 22) and (b) that they wanted to eliminate the question fonnat 
presented to them (3.3% overall; also 3.4% for U.S.-only citizens; see Table 24). For Fonnat C, 
non-U.S. citizens particularly expressed a desire to be able to report additional nationalities and 
ancestries (8.5%/5.4%; the percentages for U.S.-only citizens were lower at 4.6%/1.0%; see 
Table 24). One non-U.S citizen went as far as saying, “I think that instead of having questions 
based on race, it should suffice to have questions based on nationality only.” This desire on the 
part of non-U.S. citizens to identify nationality (e.g., Brazilian) in lieu of race (e.g., Black) raises 
a question about whether U.S.-style questions about race and ethnicity are appropriate for non- 
U.S. citizens. To sum up, the open-ended nature of the responses makes interpretation of the 
practical significance of the issues raised unclear. This ambiguity notwithstanding, the most 
pervasively raised concerns focused on what to select if one identifies with more than one race or 
if one’s race and/or ethnicity is not included in the response options (e.g., Middle Eastern). The 
other trends affected Hispanic respondents; participants did not understand how to identify race 
in the second question if they identified as Hispanic in the first question. 
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Issues Raised With Regard to the International-Style Questions 

Because we designed the international-style questions (Figures 1-11) primarily for a non- 
U.S. population, we focused our analyses of these questions on non-U.S. citizens. An open- 
ended item requesting feedback about these international-style questions appearing in the bottom 
half of Figure 16 states, “We also would appreciate below your general comments about this 
survey and the other response choices in it.” Below this statement appeared a hyperlink that 
displayed on one screen just the international-style questions (see Figure 16). In response to the 
open-ended item, only 29 (12%) of the 245 non-U. S. citizens in this study provided any 
comments, and many of the comments did not apply to the international-style questions in 
particular. It is unclear whether this low response rate indicates satisfaction with the 
international-style questions, apathy toward the open-ended item, a belief that previous questions 
had already captured participants’ concerns, and/or other factors. Those who did reply raised the 
following themes: (a) how to deal with overlaps and inconsistencies among the concepts of 
citizenship, ethnicity, and race (n = 16); (b) concerns about the final survey asking for 
infonnation about parents and guardians (n = 9; see Figures 7-8); (c) the underinclusion of 
response options for languages (n = 4; see Figure 10); (d) U.S. residency issues (n = 3); (e) other 
issues about ethnicity and race (n = 2); (f) other issues about citizenship (n = 2); and (g) an issue 
about parents/guardians and language (n = 1). (Note that total N is larger than 29, because some 
respondents provided more than one comment.) 

We are unable to distill down the issues raised with regard to the international-style 
questions, because those comments were infrequent, nonspecific, conflicting, and/or abstruse. 

The issues about overlaps and inconsistencies among citizenship, ethnicity, and race that 
respondents raised did not focus on the international-style questions specifically. Rather, they 
tended to be more general and included contradictory reactions about whether ETS should ask 
about race, ethnicity, and/or citizenship; whether those concepts should be narrowly or broadly 
defined; whether the survey questions were specific enough for citizens of African nations; and 
whether ETS should have a separate question not just for Hispanic registrants (e.g., one also for 
Arabic registrants). Five out of the nine comments about the survey questions concerning parents 
and guardians (see Figures 7-8) indicated confusion and concern about the propriety and purpose 
of those questions. Comments about the language question (Figure 10) generally consisted of 
requests for response options for additional languages (e.g., Nepal Bhasa/Newari). The 
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remaining comments lacked common themes. On the whole, the comments about the 
international-style questions suggest that asking about race and ethnicity to an international 
subpopulation involves complex issues that may warrant further investigation. That being said, 
the international-style questions that we asked (Figures 1-11) did not prompt non-U.S. citizens 
to express many concerns. 


Conclusion 

The authors believe that the BIQ question about race and ethnicity should be changed, 
because the known benefits of changing the question outweigh the known costs. Although the 
purpose of this study is not to conduct a legal or business analysis, we nevertheless reiterate that 
the 1997 Standards that the DOE implemented might apply to ETS—and even if they do not, it 
may make good business sense to comply with them anyway. From a more empirical 
perspective, U.S.-acculturated and multiracial registrants prefer a more pluralistic question 
(Fonnat C) than the one that the BIQ currently uses (Format A). U.S.-acculturated respondents 
comprise the majority of the GRE General Test registrants, and the multiracial population in the 
United States is growing rapidly. Monoracial Asian respondents and monoracial American 
Indian/Native Alaskan respondents wanted to be able to report national origins and tribal 
affiliations unavailable on the BIQ. The most common comments about Formats A, B, and C 
were about the desire to report racial and ethnic designations that not even Fonnat C explicitly 
provided; equally common for Fonnats A and B were desires to be able to report more than one 
race. Furthermore, the vast majority of non-U.S. citizens did not express any objections to the 
international-style questions, most of which do not appear in the current BIQ. To the extent that 
non-U.S. citizens did comment on the international-style questions, their remarks were mixed 
and sometimes contradictory. 

We recommend that the question about race and ethnicity be changed to a format that 
shares many of the aspects of Format C but is not identical to it. Among the three fonnats—A, B, 
and C—only Fonnat C satisfies the requirements of the 1997 Standards and many of the desired 
specifications that participants expressed in this study. Contrary to the 1997 Standards, Format A 
(which is virtually identical to the current BIQ format) combines the Hispanic and race questions 
into a single question, and neither Fonnat A nor B pennits multiracial responding. In addition, 
only Fonnat C provides Asian respondents and American Indians/Native Alaskan respondents 
with the opportunity to report national origin or tribal affiliation. However, some respondents 


50 



desired the ability to report national/regional origins not explicitly provided in Fonnat C (e.g., 
Middle Eastern). Furthermore, several Hispanic U.S. citizens often thought of themselves as just 
Hispanic and not in terms of race as classified in Formats B and C. Based on the 1997 Standards 
and opinions of the respondents in our study, we recommend that the current BIQ question about 
race and ethnicity be replaced with one that has the following characteristics: 

1. It includes a two-question fonnat for Hispanic origin and race as defined by U.S. 
standards. Formats B and C illustrate this two-question fonnat. We recommend that 
the Hispanic-origin question omit the terms Latino and Spanish from its stem except 
to clarify the term Hispanic. Specifically, we recommend that instead of “Are you of 
Hispanic, Latino, or Spanish origin?” the stem read, “Are you of Hispanic origin? 
People who identify as Hispanic were born in or have family from a Latin American 
or Caribbean country in which Spanish is the main language.” Although participants 
did not generally complain about the grouping Hispanic, Latino, or Spanish, past 
research cited by Martin (2006) indicates that some respondents to surveys using 
these labels misinterpret Latino as including all nations with languages that have 
Latin roots and Spanish as including only those born in Spain (Gerber & Crowley, 
2005). In fact, one of the participants in the present study remarked, “From question 
about what Hispanic, Latino, or Spanish origin. I am considered Latino, because 
Brazil is part of Latin America, however not from Spanish origin (Portuguese 
origin).” 

2. The Hispanic-origin question is positioned as the first of these two questions, with the 
racial question coming second. Fonnats B and C illustrate this ordering. 

3. The following are provided as main choices in the race question: (a) American Indian 
or Alaska Native, (b) Asian, (c) Black or African American, (d) Native Hawaiian or 
Other Pacific Islander, and (e) White. Fonnats A, B, and C provide these options. 
Although these options are not alphabetized on the 2010 U.S. Census fonn, some 
respondents indicated that alphabetical ordering would seem to be more neutral. 

4. The Some Other Race option is retained from the 2010 U.S. Census form (Format C) 
but with the option shortened to Other Race because Some may make the option seem 
pejorative. In this study, some participants expressed discomfort over the option 
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Some Other Race and requested that the word Some be removed (0.8% overall; 0.8% 
of U.S.-only citizens; and 1.4% of dual U.S. citizens; see Tables 23 and 24). The 
extent to which removing the word Some would alleviate this discomfort remains 
unclear, because some registrants might dislike the option for other reasons. 
Historically, Hispanic respondents have been the most frequent choosers of the Some 
Other Race option on the U.S. Census form (see Recommendations From the 
Interagency Committee, 1997). Therefore, the provision of a separate prior question 
for Hispanic origin combined with the proposed changes described in Points 6 and 7 
might, on their own, reduce dissatisfaction with the Some Other Race option. 

5. Respondents are permitted to select more than one race in the racial question. Format 
C illustrates this flexibility. 

6. A response option is provided for the race question, “I do not identify with any of the 
choices for this question.” 

7. American Indians/Alaskan Natives are pennitted to input a tribal affiliation. 

8. All respondents, not just Asian respondents, are permitted to select national origin 
(which includes birth country) from a drop-down list of countries. This might require 
the creation of a third question, which would supplement the requirements of the 
1997 Standards. Because many people identify with more than one national origin 
(e.g., father from Kenya, mother from England), the question could pennit more than 
one response to national origin. We recognize that these additional responses do not 
necessarily facilitate the identification of groups for which DIF or DP would be 
required or necessary. However, there is some evidence that these identifications 
would be helpful to fairness analyses. As already discussed above, Black respondents 
bom in Africa, Latin America, or the Caribbean or who would identify a national 
origin there experience better educational outcomes than African Americans whose 
families have lived in the United States for several generations. Also, there is 
nontrivial variability in educational expectations and attainment of Asian Americans 
based upon national origin (see, e.g., Sakamoto, Goyette, & Kim, 2009). Goyette and 
Xie (1999) reported that the expectation of graduation from college is almost 30% 
higher for South Asian American students than for Southeast Asian American 
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students. It is because of such discrepancies that Asian respondents would benefit 
from fairness analyses made possible by identification of national origin. 

Figure 17 illustrates the U.S.-style questions that we recommend. 


Areyou-of-Hispanic-origin?-People-whoidentify-asHispanic-were-born-in-or-have-family- 
from-a-Latm-American-or-Caribbean-country-in -which •Spanishtslhemainlanguage. r 

□ No. not of Hispanic origin'll 

□ Yes.Mexican. Mexican American. Chicano a^f 

□ Y es, Puerto Ricanl 

□ Yes.Cuban c 

□ Yes.another Hispanic otiem—Type-in-origin for example, -.Argentinean, - Colombian, ■ 
Dominican -Kkaraguan, Sahadoran -andso-on 

[FREE RESPONSE TEXT BOX COES HERE]* - 

«T 

Wh at is your-race? Mark E one or more boxes.][ 

□ American Indian orAlaskaNative' Type-in-name-of-enro lied-or-principal-tribe- l r 
[FREE RESPONSE TEXT BOX COES HERE]* - 

□ Asian*! 

□ Black or African American* - 

□ Native Hawaiian or Odier Pacific Islander*" 

□ White* - 

□ Other-race— 

[FREE RESPONSE TEXT BOX COES HERE]*; 

□ I do not identify with any of the choices for this question.^ 

€T 

Wh a t is-youraneestry?For-many res pon den ts.-aneestryis-different-from-citizen ship.- 
An ces try often-describes-a-country-from-which-a-person's-family-or-ancestors-immigrated- 
toihe-United-States. You-may-make-up io-twofselections.-You-may-skip this-question.* - 

PROP-DOWN ALPHABETICALMENU!. WITH "I DO NOT WISH TO RESPOND -- AND 
“IDO NOT KNOW' AS THE FIRST TWO OPTIONS AND “UNITED STATES" AND 
“NONEOF THE ABOVE’ LISTED LAST] 1 

PROPPOWN ALPHABETICAL MENU 2. WITH ‘TDO NOT WISH TO RESPOND” AND 
“I DO NOTKNOW” AS THE FIRST TWO OPTIONS AND “UNITED STATES’’ AND 
‘NONEOF THE ABOVE” LISTED -LASTTlf 


Figure 17. Recommended format for U.S.-style question about race and ethnicity. 


With the exception of U.S. permanent residents, evidence suggests that there would be 
little or no benefit to allowing non-U.S. citizens to respond to U.S.-style questions about 
ethnicity and race. We lack enough data to offer an opinion about whether ETS should add 
response options for groups that span several countries (e.g., Middle Eastern/Arabic, Hmong, 
Kurdish) or are ethnic minorities in a single foreign country (e.g., the Karen people in 
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Myanmar). The U.S. Census Bureau has considered adding a Middle Eastern/Arabic category, 
but the 1997 Standards did not advocate this decision due to difficulties encountered in defining 
such a category (see Revisions to the Standards for the Classification of Federal Data on Race 
and Ethnicity, 1997), and the 2010 U.S. Census form lacked this option. Allowing U.S. citizens 
to select a national origin and permanent U.S. residents to select a citizenship may resolve some 
aspects of this issue, because national origins and citizenships can be aggregated to form these 
transnational groups. However, some groups are minorities within the single country in which 
they primarily reside (e.g., the Karen people in Myanmar). GRE sample sizes for these minorities 
are likely to be extremely small or nonexistent, so we do not consider this to be a practical issue. 

In addition, we believe that the international-style questions in the final survey, if 
included in the BIQ, would require some modification. We see no reasons to exclude U.S. 
citizens from answering these questions. Given that 72 (16%) of the 460 U.S. citizens in this 
study possessed dual citizenship, we believe that eliciting information about a second citizenship 
is important. (The current BIQ asks for only a single citizenship.) GRE General Test items might 
not function or predict the same scores for U.S.-only citizens and dual U.S. citizens because of 
cultural, linguistic, and educational differences between the two groups. Citizenship (especially 
if it is dual citizenship) does not always clarify the country in which one was raised, and where 
one was raised can affect how an assessment functions or predicts for that person. Therefore, 
either a question about one’s country of birth (like Figure 6) or the country in which one was 
raised seems appropriate. Some non-U.S. citizens expressed concerns about questions regarding 
parents, guardians, and caregivers. It is possible that national origin questions can adequately 
substitute for questions about parent/guardian/caregiver country of birth (see Figures 7-8). 
(Although this substitution might avoid confusion for questions about race/ethnicity, given the 
complexity of family structures it does not resolve the issue of how best to seek information 
about parents for other questions in the BIQ.) As discussed previously, questions about English 
language proficiency (see Figures 9-11) are important, because not only can English language 
proficiency affect the functioning and prediction of assessments, but these questions can also 
help identify ethnic groups that otherwise might go undetected (e.g., French-speaking Canadians 
versus English-speaking Canadians). It is important to know the nature of a registrant’s 
familiarity with English (writing, speaking, etc.), because reading is the skill relevant to 
performance on the Verbal and Quantitative Reasoning sections of the GRE General Test, and 
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writing ability is the skill relevant to performance on the Analytical Writing section. The current 
BIQ makes no such distinctions among language skills. Knowing the non-English languages 
with which a registrant is familiar can also inform how to conduct DIF and DP analyses, because 
some languages (e.g., Romance languages) can facilitate comprehension of English, whereas 
other languages would not (e.g., Chinese). It might be helpful to categorize registrants by 
language families to look at DIF and DP. Considering the small sample sizes involved, we 
believe that requests that additional languages spoken by relatively few be added to the list of 
languages spoken at home (see Figure 10) do not present a practical concern. Nevertheless, ETS 
could periodically add them. In conclusion, we believe that ETS should retain and/or consider 
adding the content and format of Figures 1-6 and Figures 9-11 to the BIQ. 

Further research is needed on this highly complex issue. Several of the sample sizes in 
this study are very small (including Native Hawaiian and Other Pacific Islander respondents, 
multiracial Black respondents, multiracial Asian respondents, and citizens of 
Australia/Oceania/Polynesia). Additional data collection is therefore warranted. One issue 
concerns the additional racial and ethnic groups with which respondents may wish to identify 
(e.g.. Middle Eastern/Arabic). What additional groups make sense, and how does ETS ask about 
them in a way that is comprehensible and respectful? Given that many Hispanic U.S. citizens 
have difficulty identifying with racial groups or the terms Latino and Spanish, how best do we 
frame questions about ethnicity and race for them? Are there other demographic constructs that, 
in some cases, might be no less relevant than race or ethnicity to fairness analyses (e.g., religion, 
sexual orientation)? Ultimately, it is necessary to determine the impact of these changes on DIF 
and DP analyses, which requires making determinations of what groups to create and how to 
categorize registrants who may belong to more than one group. Should a multiracial respondent 
be classified with each group with which he or she identifies, with just one group randomly 
chosen, or with no groups at all? Theory, legal requirements, political necessity, economics, 
practicality, statistical concerns, personal values, and/or mere curiosity may drive decisions. 
Among other concerns, dividing registrants into additional groups for a single analysis risks 
reducing group sizes to the point where the possibility of sampling error becomes large. 
Additional analyses also increase the risk of finding statistically and practically significant effect 
sizes by mere chance. 
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Notes 


1 This study is intended to be a descriptive investigation. We did not conduct statistical 

significance testing of results, because many of the sample sizes for racial and ethnic groups 
would result in low statistical power. 

'j 

Unfortunately, measurement of the construct U.S.-acculturated is not crystal clear, partly 
because it can be based on citizenship, residency, and/or birthplace (among other things). 
Citizenship and birthplace are not always clearly distinguishable, because legally everyone 
bom in the United States is automatically a U.S. citizen, and relatively few people who have 
U.S. citizenship give up that citizenship. Also, there is a difference between (a) birthplace and 
citizenship and (b) where someone was raised (and thus socialized). For the purposes of this 
study, U.S.-acculturated test-takers include those who are U.S. citizens, born in the United 
States, and/or permanent U.S. residents. 


59 



GRE-ETS 
PO Box 6000 
Princeton, NJ 08541-6000 
USA 


To obtain more information about GRE 
programs and services, use one of the following: 
Phone: 1-866-473-4373 
(U.S., U.S. Territories*, and Canada) 
1-609-771-7670 
(all other locations) 

Web site: www.gre.org 


* America Samoa, Guam, Puerto Rico, and US Virgin Islands 



